[PATCH 00/21] git p4: work on cygwin

[PATCH 00/21] git p4: work on cygwin

[Pete Wyckoff]

This series fixes problems in git-p4, and its tests, so that
git-p4 works on the cygwin platform.

See the wiki for info on how to get started on cygwin:

    https://git.wiki.kernel.org/index.php/GitP4

Testing by people who use cygwin would be appreciated.  It would
be good to support cygwin more regularly.  Anyone who had time
to contribute to testing on cygwin, and reporting problems, would
be welcome.

There's more work requried to support msysgit.  Those patches
are not in good enough shape to ship out yet, but a lot of what
is in this series is required for msysgit too.

These patches:

    - fix bugs in git-p4 related to issues found on cygwin
    - cleanup some ugly code in git-p4 observed in error paths while
      getting tests to work on cygwin
    - simplify and refactor code and tests to make cygwin changes easier
    - handle newline and path issues for cygwin platform
    - speed up some aspects of git-p4 by removing extra shell invocations

Pete Wyckoff (21):
  git p4: temp branch name should use / even on windows
  git p4: remove unused imports
  git p4: generate better error message for bad depot path
  git p4: fix error message when "describe -s" fails
  git p4 test: use client_view to build the initial client
  git p4 test: use client_view in t9806
  git p4 test: start p4d inside its db dir
  git p4 test: translate windows paths for cygwin
  git p4: remove unreachable windows \r\n conversion code
  git p4: scrub crlf for utf16 files on windows
  git p4 test: newline handling
  git p4 test: use LineEnd unix in windows tests too
  git p4 test: avoid wildcard * in windows
  git p4: cygwin p4 client does not mark read-only
  git p4 test: disable chmod test for cygwin
  git p4: disable read-only attribute before deleting
  git p4: avoid shell when mapping users
  git p4: avoid shell when invoking git rev-list
  git p4: avoid shell when invoking git config --get-all
  git p4: avoid shell when calling git config
  git p4: introduce gitConfigBool

 git-p4.py                     | 122 ++++++++++++++++++++++++++++--------------
 t/lib-git-p4.sh               |  60 +++++++++++++++------
 t/t9800-git-p4-basic.sh       |   5 ++
 t/t9802-git-p4-filetype.sh    | 117 ++++++++++++++++++++++++++++++++++++++++
 t/t9806-git-p4-options.sh     |  50 ++++++++---------
 t/t9807-git-p4-submit.sh      |  14 ++++-
 t/t9809-git-p4-client-view.sh |  14 +++--
 t/t9812-git-p4-wildcards.sh   |  37 ++++++++++---
 t/t9815-git-p4-submit-fail.sh |   4 +-
 t/test-lib.sh                 |   3 ++
 10 files changed, 330 insertions(+), 96 deletions(-)

-- 
1.7.12.1.403.g28165e1

[PATCH 01/21] git p4: temp branch name should use / even on windows

[Pete Wyckoff]

Commit fed2369 (git-p4: Search for parent commit on branch creation,
2012-01-25) uses temporary branches to help find the parent of a
new p4 branch.  The temp branches are of the form "git-p4-tmp/%d"
for some p4 change number.  Mistakenly, this string was made
using os.path.join() instead of just string concatenation.  On
windows, this turns into a backslash (\), which is not allowed in
git branch names.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index 882b1bb..1e7a22a 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2599,7 +2599,7 @@ class P4Sync(Command, P4UserMap):
 
                         blob = None
                         if len(parent) > 0:
-                            tempBranch = os.path.join(self.tempBranchLocation, "%d" % (change))
+                            tempBranch = "%s/%d" % (self.tempBranchLocation, change)
                             if self.verbose:
                                 print "Creating temporary branch: " + tempBranch
                             self.commit(description, filesForCommit, tempBranch)
-- 
1.7.12.1.403.g28165e1

[PATCH 02/21] git p4: remove unused imports

[Pete Wyckoff]

Found by "pyflakes" checker tool.
Modules shelve, getopt were unused.
Module os.path is exported by os.
Reformat one-per-line as is PEP008 suggested style.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 1e7a22a..97699ef 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -7,10 +7,16 @@
 #            2007 Trolltech ASA
 # License: MIT <http://www.opensource.org/licenses/mit-license.php>
 #
-
-import optparse, sys, os, marshal, subprocess, shelve
-import tempfile, getopt, os.path, time, platform
-import re, shutil
+import sys
+import os
+import optparse
+import marshal
+import subprocess
+import tempfile
+import time
+import platform
+import re
+import shutil
 
 verbose = False
 
-- 
1.7.12.1.403.g28165e1

[PATCH 03/21] git p4: generate better error message for bad depot path

[Pete Wyckoff]

Depot paths must start with //.  Exit with a better explanation
when a bad depot path is supplied.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py               | 1 +
 t/t9800-git-p4-basic.sh | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/git-p4.py b/git-p4.py
index 97699ef..eef5c94 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -3035,6 +3035,7 @@ class P4Clone(P4Sync):
         self.cloneExclude = ["/"+p for p in self.cloneExclude]
         for p in depotPaths:
             if not p.startswith("//"):
+                sys.stderr.write('Depot paths must start with "//": %s\n' % p)
                 return False
 
         if not self.cloneDestination:
diff --git a/t/t9800-git-p4-basic.sh b/t/t9800-git-p4-basic.sh
index b7ad716..c5f4c88 100755
--- a/t/t9800-git-p4-basic.sh
+++ b/t/t9800-git-p4-basic.sh
@@ -30,6 +30,11 @@ test_expect_success 'basic git p4 clone' '
 	)
 '
 
+test_expect_success 'depot typo error' '
+	test_must_fail git p4 clone --dest="$git" /depot 2>errs &&
+	grep -q "Depot paths must start with" errs
+'
+
 test_expect_success 'git p4 clone @all' '
 	git p4 clone --dest="$git" //depot@all &&
 	test_when_finished cleanup_git &&
-- 
1.7.12.1.403.g28165e1

[PATCH 04/21] git p4: fix error message when "describe -s" fails

[Pete Wyckoff]

The output was a bit nonsensical, including a bare %d.  Fix it
to make it easier to understand.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index eef5c94..d7ee4b4 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2679,7 +2679,8 @@ class P4Sync(Command, P4UserMap):
             if r.has_key('time'):
                 newestTime = int(r['time'])
         if newestTime is None:
-            die("\"describe -s\" on newest change %d did not give a time")
+            die("Output from \"describe -s\" on newest change %d did not give a time" %
+                newestRevision)
         details["time"] = newestTime
 
         self.updateOptionDict(details)
-- 
1.7.12.1.403.g28165e1

[PATCH 05/21] git p4 test: use client_view to build the initial client

[Pete Wyckoff]

Simplify the code a bit by using an existing function.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index 7061dce..890ee60 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -74,15 +74,8 @@ start_p4d() {
 	fi
 
 	# build a client
-	(
-		cd "$cli" &&
-		p4 client -i <<-EOF
-		Client: client
-		Description: client
-		Root: $cli
-		View: //depot/... //client/...
-		EOF
-	)
+	client_view "//depot/... //client/..." &&
+
 	return 0
 }
 
-- 
1.7.12.1.403.g28165e1

[PATCH 06/21] git p4 test: use client_view in t9806

[Pete Wyckoff]

Use the standard client_view function from lib-git-p4.sh
instead of building one by hand.  This requires a bit of
rework, using the current value of $P4CLIENT for the client
name.  It also reorganizes the test to isolate changes to
$P4CLIENT and $cli in a subshell.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh           |  4 ++--
 t/t9806-git-p4-options.sh | 50 ++++++++++++++++++++++-------------------------
 2 files changed, 25 insertions(+), 29 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index 890ee60..d558dd0 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -116,8 +116,8 @@ marshal_dump() {
 client_view() {
 	(
 		cat <<-EOF &&
-		Client: client
-		Description: client
+		Client: $P4CLIENT
+		Description: $P4CLIENT
 		Root: $cli
 		View:
 		EOF
diff --git a/t/t9806-git-p4-options.sh b/t/t9806-git-p4-options.sh
index fa40cc8..37ca30a 100755
--- a/t/t9806-git-p4-options.sh
+++ b/t/t9806-git-p4-options.sh
@@ -126,37 +126,33 @@ test_expect_success 'clone --use-client-spec' '
 		exec >/dev/null &&
 		test_must_fail git p4 clone --dest="$git" --use-client-spec
 	) &&
-	cli2=$(test-path-utils real_path "$TRASH_DIRECTORY/cli2") &&
+	# build a different client
+	cli2="$TRASH_DIRECTORY/cli2" &&
 	mkdir -p "$cli2" &&
 	test_when_finished "rmdir \"$cli2\"" &&
-	(
-		cd "$cli2" &&
-		p4 client -i <<-EOF
-		Client: client2
-		Description: client2
-		Root: $cli2
-		View: //depot/sub/... //client2/bus/...
-		EOF
-	) &&
-	P4CLIENT=client2 &&
 	test_when_finished cleanup_git &&
-	git p4 clone --dest="$git" --use-client-spec //depot/... &&
-	(
-		cd "$git" &&
-		test_path_is_file bus/dir/f4 &&
-		test_path_is_missing file1
-	) &&
-	cleanup_git &&
-
-	# same thing again, this time with variable instead of option
 	(
-		cd "$git" &&
-		git init &&
-		git config git-p4.useClientSpec true &&
-		git p4 sync //depot/... &&
-		git checkout -b master p4/master &&
-		test_path_is_file bus/dir/f4 &&
-		test_path_is_missing file1
+		# group P4CLIENT and cli changes in a sub-shell
+		P4CLIENT=client2 &&
+		cli="$cli2" &&
+		client_view "//depot/sub/... //client2/bus/..." &&
+		git p4 clone --dest="$git" --use-client-spec //depot/... &&
+		(
+			cd "$git" &&
+			test_path_is_file bus/dir/f4 &&
+			test_path_is_missing file1
+		) &&
+		cleanup_git &&
+		# same thing again, this time with variable instead of option
+		(
+			cd "$git" &&
+			git init &&
+			git config git-p4.useClientSpec true &&
+			git p4 sync //depot/... &&
+			git checkout -b master p4/master &&
+			test_path_is_file bus/dir/f4 &&
+			test_path_is_missing file1
+		)
 	)
 '
 
-- 
1.7.12.1.403.g28165e1

[PATCH 07/21] git p4 test: start p4d inside its db dir

[Pete Wyckoff]

This will avoid having to do native path conversion for
windows.  Also may be a bit cleaner always to know that p4d
has that working directory, instead of wherever the function
was called from.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index d558dd0..402d736 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -40,8 +40,11 @@ start_p4d() {
 	mkdir -p "$db" "$cli" "$git" &&
 	rm -f "$pidfile" &&
 	(
-		p4d -q -r "$db" -p $P4DPORT &
-		echo $! >"$pidfile"
+		cd "$db" &&
+		{
+			p4d -q -p $P4DPORT &
+			echo $! >"$pidfile"
+		}
 	) &&
 
 	# This gives p4d a long time to start up, as it can be
-- 
1.7.12.1.403.g28165e1

[PATCH 08/21] git p4 test: translate windows paths for cygwin

[Pete Wyckoff]

Native windows binaries do not understand posix-like
path mapping offered by cygwin.  Convert paths to native
using "cygpath --windows" before presenting them to p4d.

This is done using the AltRoots mechanism of p4.  Both the
posix and windows forms are put in the client specification,
allowing p4 to find its location by native path even though
the environment reports a different PWD.

Shell operations in tests will use the normal form of $cli,
which will look like a posix path in cygwin, while p4 will
use AltRoots to match against the windows form of the working
directory.

Thanks-to: Sebastian Schuberth <sschuberth@gmail.com>
Thanks-to: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh | 24 ++++++++++++++++++++++--
 t/test-lib.sh   |  3 +++
 2 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index 402d736..e2941ac 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -8,7 +8,8 @@ TEST_NO_CREATE_REPO=NoThanks
 
 . ./test-lib.sh
 
-if ! test_have_prereq PYTHON; then
+if ! test_have_prereq PYTHON
+then
 	skip_all='skipping git p4 tests; python not available'
 	test_done
 fi
@@ -17,6 +18,24 @@ fi
 	test_done
 }
 
+# On cygwin, the NT version of Perforce can be used.  When giving
+# it paths, either on the command-line or in client specifications,
+# be sure to use the native windows form.
+#
+# Older versions of perforce were available compiled natively for
+# cygwin.  Those do not accept native windows paths, so make sure
+# not to convert for them.
+native_path() {
+	path="$1" &&
+	if test_have_prereq CYGWIN && ! p4 -V | grep -q CYGWIN
+	then
+		path=$(cygpath --windows "$path")
+	else
+		path=$(test-path-utils real_path "$path")
+	fi &&
+	echo "$path"
+}
+
 # Try to pick a unique port: guess a large number, then hope
 # no more than one of each test is running.
 #
@@ -32,7 +51,7 @@ P4EDITOR=:
 export P4PORT P4CLIENT P4EDITOR
 
 db="$TRASH_DIRECTORY/db"
-cli=$(test-path-utils real_path "$TRASH_DIRECTORY/cli")
+cli="$TRASH_DIRECTORY/cli"
 git="$TRASH_DIRECTORY/git"
 pidfile="$TRASH_DIRECTORY/p4d.pid"
 
@@ -122,6 +141,7 @@ client_view() {
 		Client: $P4CLIENT
 		Description: $P4CLIENT
 		Root: $cli
+		AltRoots: $(native_path "$cli")
 		View:
 		EOF
 		for arg ; do
diff --git a/t/test-lib.sh b/t/test-lib.sh
index f8e3733..fd04870 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -624,12 +624,14 @@ case $(uname -s) in
 	# backslashes in pathspec are converted to '/'
 	# exec does not inherit the PID
 	test_set_prereq MINGW
+	test_set_prereq NOT_CYGWIN
 	test_set_prereq SED_STRIPS_CR
 	;;
 *CYGWIN*)
 	test_set_prereq POSIXPERM
 	test_set_prereq EXECKEEPSPID
 	test_set_prereq NOT_MINGW
+	test_set_prereq CYGWIN
 	test_set_prereq SED_STRIPS_CR
 	;;
 *)
@@ -637,6 +639,7 @@ case $(uname -s) in
 	test_set_prereq BSLASHPSPEC
 	test_set_prereq EXECKEEPSPID
 	test_set_prereq NOT_MINGW
+	test_set_prereq NOT_CYGWIN
 	;;
 esac
 
-- 
1.7.12.1.403.g28165e1

[PATCH 09/21] git p4: remove unreachable windows \r\n conversion code

[Pete Wyckoff]

Replacing \r\n with \n on windows was added in c1f9197 (Replace
\r\n with \n when importing from p4 on Windows, 2007-05-24), to
work around an oddity with "p4 print" on windows.  Text files
are printed with "\r\r\n" endings, regardless of whether they
were created on unix or windows, and regardless of the client
LineEnd setting.

As of d2c6dd3 (use p4CmdList() to get file contents in Python
dicts. This is more robust., 2007-05-23), git-p4 uses "p4 -G
print", which generates files in a raw format.  As the native
line ending format if p4 is \n, there will be no \r\n in the
raw text.

Actually, it is possible to generate a text file so that the
p4 representation includes embedded \r\n, even though this is not
normal on either windows or unix.  In that case the code would
have mistakenly stripped them out, but now they will be left
intact.

More information on how p4 deals with line endings is here:

    http://kb.perforce.com/article/63

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index d7ee4b4..b773b09 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2064,15 +2064,6 @@ class P4Sync(Command, P4UserMap):
             print "\nIgnoring apple filetype file %s" % file['depotFile']
             return
 
-        # Perhaps windows wants unicode, utf16 newlines translated too;
-        # but this is not doing it.
-        if self.isWindows and type_base == "text":
-            mangled = []
-            for data in contents:
-                data = data.replace("\r\n", "\n")
-                mangled.append(data)
-            contents = mangled
-
         # Note that we do not try to de-mangle keywords on utf16 files,
         # even though in theory somebody may want that.
         pattern = p4_keywords_regexp_for_type(type_base, type_mods)
-- 
1.7.12.1.403.g28165e1

[PATCH 10/21] git p4: scrub crlf for utf16 files on windows

[Pete Wyckoff]

Files of type utf16 are handled with "p4 print" instead of the
normal "p4 -G print" interface due to how the latter does not
produce correct output.  See 55aa571 (git-p4: handle utf16
filetype properly, 2011-09-17) for details.

On windows, though, "p4 print" can not be told which line
endings to use, as there is no underlying client, and always
chooses crlf, even for utf16 files.  Convert the \r\n into \n
when importing utf16 files.

The fix for this is complex, in that the problem is a property
of the NT version of p4.  There are old versions of p4 that
were compiled directly for cygwin that should not be subjected
to text replacement.  The right check here, then, is to look
at the p4 version, not the OS version.  Note also that on cygwin,
platform.system() is "CYGWIN_NT-5.1" or similar, not "Windows".

Add a function to memoize the p4 version string and use it to
check for "/NT", indicating the Windows build of p4.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index b773b09..5b2f73d 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -147,6 +147,22 @@ def p4_system(cmd):
     expand = isinstance(real_cmd, basestring)
     subprocess.check_call(real_cmd, shell=expand)
 
+_p4_version_string = None
+def p4_version_string():
+    """Read the version string, showing just the last line, which
+       hopefully is the interesting version bit.
+
+       $ p4 -V
+       Perforce - The Fast Software Configuration Management System.
+       Copyright 1995-2011 Perforce Software.  All rights reserved.
+       Rev. P4/NTX86/2011.1/393975 (2011/12/16).
+    """
+    global _p4_version_string
+    if not _p4_version_string:
+        a = p4_read_pipe_lines(["-V"])
+        _p4_version_string = a[-1].rstrip()
+    return _p4_version_string
+
 def p4_integrate(src, dest):
     p4_system(["integrate", "-Dt", wildcard_encode(src), wildcard_encode(dest)])
 
@@ -1903,7 +1919,6 @@ class P4Sync(Command, P4UserMap):
         self.syncWithOrigin = True
         self.importIntoRemotes = True
         self.maxChanges = ""
-        self.isWindows = (platform.system() == "Windows")
         self.keepRepoPath = False
         self.depotPaths = None
         self.p4BranchesInGit = []
@@ -2048,7 +2063,14 @@ class P4Sync(Command, P4UserMap):
             # operations.  utf16 is converted to ascii or utf8, perhaps.
             # But ascii text saved as -t utf16 is completely mangled.
             # Invoke print -o to get the real contents.
+            #
+            # On windows, the newlines will always be mangled by print, so put
+            # them back too.  This is not needed to the cygwin windows version,
+            # just the native "NT" type.
+            #
             text = p4_read_pipe(['print', '-q', '-o', '-', file['depotFile']])
+            if p4_version_string().find("/NT") >= 0:
+                text = text.replace("\r\n", "\n")
             contents = [ text ]
 
         if type_base == "apple":
-- 
1.7.12.1.403.g28165e1

[PATCH 11/21] git p4 test: newline handling

[Pete Wyckoff]

P4 stores newlines in the depos as \n.  By default, git does this
too, both on unix and windows.  Test to make sure that this stays
true.

Both git and p4 have mechanisms to use \r\n in the working
directory.  Exercise these.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/t9802-git-p4-filetype.sh | 117 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 117 insertions(+)

diff --git a/t/t9802-git-p4-filetype.sh b/t/t9802-git-p4-filetype.sh
index 21924df..c5ab626 100755
--- a/t/t9802-git-p4-filetype.sh
+++ b/t/t9802-git-p4-filetype.sh
@@ -8,6 +8,123 @@ test_expect_success 'start p4d' '
 	start_p4d
 '
 
+#
+# This series of tests checks newline handling  Both p4 and
+# git store newlines as \n, and have options to choose how
+# newlines appear in checked-out files.
+#
+test_expect_success 'p4 client newlines, unix' '
+	(
+		cd "$cli" &&
+		p4 client -o | sed "/LineEnd/s/:.*/:unix/" | p4 client -i &&
+		printf "unix\ncrlf\n" >f-unix &&
+		printf "unix\r\ncrlf\r\n" >f-unix-as-crlf &&
+		p4 add -t text f-unix &&
+		p4 submit -d f-unix &&
+
+		# LineEnd: unix; should be no change after sync
+		cp f-unix f-unix-orig &&
+		p4 sync -f &&
+		test_cmp f-unix-orig f-unix &&
+
+		# make sure stored in repo as unix newlines
+		# use sed to eat python-appened newline
+		p4 -G print //depot/f-unix | marshal_dump data 2 |\
+		    sed \$d >f-unix-p4-print &&
+		test_cmp f-unix-orig f-unix-p4-print &&
+
+		# switch to win, make sure lf -> crlf
+		p4 client -o | sed "/LineEnd/s/:.*/:win/" | p4 client -i &&
+		p4 sync -f &&
+		test_cmp f-unix-as-crlf f-unix
+	)
+'
+
+test_expect_success 'p4 client newlines, win' '
+	(
+		cd "$cli" &&
+		p4 client -o | sed "/LineEnd/s/:.*/:win/" | p4 client -i &&
+		printf "win\r\ncrlf\r\n" >f-win &&
+		printf "win\ncrlf\n" >f-win-as-lf &&
+		p4 add -t text f-win &&
+		p4 submit -d f-win &&
+
+		# LineEnd: win; should be no change after sync
+		cp f-win f-win-orig &&
+		p4 sync -f &&
+		test_cmp f-win-orig f-win &&
+
+		# make sure stored in repo as unix newlines
+		# use sed to eat python-appened newline
+		p4 -G print //depot/f-win | marshal_dump data 2 |\
+		    sed \$d >f-win-p4-print &&
+		test_cmp f-win-as-lf f-win-p4-print &&
+
+		# switch to unix, make sure lf -> crlf
+		p4 client -o | sed "/LineEnd/s/:.*/:unix/" | p4 client -i &&
+		p4 sync -f &&
+		test_cmp f-win-as-lf f-win
+	)
+'
+
+test_expect_success 'ensure blobs store only lf newlines' '
+	test_when_finished cleanup_git &&
+	(
+		cd "$git" &&
+		git init &&
+		git p4 sync //depot@all &&
+
+		# verify the files in .git are stored only with newlines
+		o=$(git ls-tree p4/master -- f-unix | cut -f1 | cut -d\  -f3) &&
+		git cat-file blob $o >f-unix-blob &&
+		test_cmp "$cli"/f-unix-orig f-unix-blob &&
+
+		o=$(git ls-tree p4/master -- f-win | cut -f1 | cut -d\  -f3) &&
+		git cat-file blob $o >f-win-blob &&
+		test_cmp "$cli"/f-win-as-lf f-win-blob &&
+
+		rm f-unix-blob f-win-blob
+	)
+'
+
+test_expect_success 'gitattributes setting eol=lf produces lf newlines' '
+	test_when_finished cleanup_git &&
+	(
+		# checkout the files and make sure core.eol works as planned
+		cd "$git" &&
+		git init &&
+		echo "* eol=lf" >.gitattributes &&
+		git p4 sync //depot@all &&
+		git checkout master &&
+		test_cmp "$cli"/f-unix-orig f-unix &&
+		test_cmp "$cli"/f-win-as-lf f-win
+	)
+'
+
+test_expect_success 'gitattributes setting eol=crlf produces crlf newlines' '
+	test_when_finished cleanup_git &&
+	(
+		# checkout the files and make sure core.eol works as planned
+		cd "$git" &&
+		git init &&
+		echo "* eol=crlf" >.gitattributes &&
+		git p4 sync //depot@all &&
+		git checkout master &&
+		test_cmp "$cli"/f-unix-as-crlf f-unix &&
+		test_cmp "$cli"/f-win-orig f-win
+	)
+'
+
+test_expect_success 'crlf cleanup' '
+	(
+		cd "$cli" &&
+		rm f-unix-orig f-unix-as-crlf &&
+		rm f-win-orig f-win-as-lf &&
+		p4 client -o | sed "/LineEnd/s/:.*/:unix/" | p4 client -i &&
+		p4 sync -f
+	)
+'
+
 test_expect_success 'utf-16 file create' '
 	(
 		cd "$cli" &&
-- 
1.7.12.1.403.g28165e1

[PATCH 12/21] git p4 test: use LineEnd unix in windows tests too

[Pete Wyckoff]

In all clients, even those created on windows, use unix line
endings.  This makes it possible to verify file contents without
doing OS-specific comparisons in all the tests.

Tests in t9802-git-p4-filetype.sh are used to make sure that
the other LineEnd options continue to work.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index e2941ac..fbd55ea 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -142,6 +142,7 @@ client_view() {
 		Description: $P4CLIENT
 		Root: $cli
 		AltRoots: $(native_path "$cli")
+		LineEnd: unix
 		View:
 		EOF
 		for arg ; do
-- 
1.7.12.1.403.g28165e1

[PATCH 13/21] git p4 test: avoid wildcard * in windows

[Pete Wyckoff]

This character is not valid in windows filenames, even though
it can appear in p4 depot paths.  Avoid using it in tests on
windows, both mingw and cygwin.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/t9809-git-p4-client-view.sh | 10 ++++++++--
 t/t9812-git-p4-wildcards.sh   | 37 +++++++++++++++++++++++++++++--------
 2 files changed, 37 insertions(+), 10 deletions(-)

diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index 281be29..fd8fa89 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -365,7 +365,10 @@ test_expect_success 'wildcard files submit back to p4, client-spec case' '
 	(
 		cd "$git" &&
 		echo git-wild-hash >dir1/git-wild#hash &&
-		echo git-wild-star >dir1/git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			echo git-wild-star >dir1/git-wild\*star
+		fi &&
 		echo git-wild-at >dir1/git-wild@at &&
 		echo git-wild-percent >dir1/git-wild%percent &&
 		git add dir1/git-wild* &&
@@ -376,7 +379,10 @@ test_expect_success 'wildcard files submit back to p4, client-spec case' '
 	(
 		cd "$cli" &&
 		test_path_is_file dir1/git-wild#hash &&
-		test_path_is_file dir1/git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			test_path_is_file dir1/git-wild\*star
+		fi &&
 		test_path_is_file dir1/git-wild@at &&
 		test_path_is_file dir1/git-wild%percent
 	) &&
diff --git a/t/t9812-git-p4-wildcards.sh b/t/t9812-git-p4-wildcards.sh
index 143d413..6763325 100755
--- a/t/t9812-git-p4-wildcards.sh
+++ b/t/t9812-git-p4-wildcards.sh
@@ -14,7 +14,10 @@ test_expect_success 'add p4 files with wildcards in the names' '
 		printf "file2\nhas\nsome\nrandom\ntext\n" >file2 &&
 		p4 add file2 &&
 		echo file-wild-hash >file-wild#hash &&
-		echo file-wild-star >file-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			echo file-wild-star >file-wild\*star
+		fi &&
 		echo file-wild-at >file-wild@at &&
 		echo file-wild-percent >file-wild%percent &&
 		p4 add -f file-wild* &&
@@ -28,7 +31,10 @@ test_expect_success 'wildcard files git p4 clone' '
 	(
 		cd "$git" &&
 		test -f file-wild#hash &&
-		test -f file-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			test -f file-wild\*star
+		fi &&
 		test -f file-wild@at &&
 		test -f file-wild%percent
 	)
@@ -40,7 +46,10 @@ test_expect_success 'wildcard files submit back to p4, add' '
 	(
 		cd "$git" &&
 		echo git-wild-hash >git-wild#hash &&
-		echo git-wild-star >git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			echo git-wild-star >git-wild\*star
+		fi &&
 		echo git-wild-at >git-wild@at &&
 		echo git-wild-percent >git-wild%percent &&
 		git add git-wild* &&
@@ -51,7 +60,10 @@ test_expect_success 'wildcard files submit back to p4, add' '
 	(
 		cd "$cli" &&
 		test_path_is_file git-wild#hash &&
-		test_path_is_file git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			test_path_is_file git-wild\*star
+		fi &&
 		test_path_is_file git-wild@at &&
 		test_path_is_file git-wild%percent
 	)
@@ -63,7 +75,10 @@ test_expect_success 'wildcard files submit back to p4, modify' '
 	(
 		cd "$git" &&
 		echo new-line >>git-wild#hash &&
-		echo new-line >>git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			echo new-line >>git-wild\*star
+		fi &&
 		echo new-line >>git-wild@at &&
 		echo new-line >>git-wild%percent &&
 		git add git-wild* &&
@@ -74,7 +89,10 @@ test_expect_success 'wildcard files submit back to p4, modify' '
 	(
 		cd "$cli" &&
 		test_line_count = 2 git-wild#hash &&
-		test_line_count = 2 git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			test_line_count = 2 git-wild\*star
+		fi &&
 		test_line_count = 2 git-wild@at &&
 		test_line_count = 2 git-wild%percent
 	)
@@ -87,7 +105,7 @@ test_expect_success 'wildcard files submit back to p4, copy' '
 		cd "$git" &&
 		cp file2 git-wild-cp#hash &&
 		git add git-wild-cp#hash &&
-		cp git-wild\*star file-wild-3 &&
+		cp git-wild#hash file-wild-3 &&
 		git add file-wild-3 &&
 		git commit -m "wildcard copies" &&
 		git config git-p4.detectCopies true &&
@@ -134,7 +152,10 @@ test_expect_success 'wildcard files submit back to p4, delete' '
 	(
 		cd "$cli" &&
 		test_path_is_missing git-wild#hash &&
-		test_path_is_missing git-wild\*star &&
+		if test_have_prereq NOT_MINGW NOT_CYGWIN
+		then
+			test_path_is_missing git-wild\*star
+		fi &&
 		test_path_is_missing git-wild@at &&
 		test_path_is_missing git-wild%percent
 	)
-- 
1.7.12.1.403.g28165e1

[PATCH 14/21] git p4: cygwin p4 client does not mark read-only

[Pete Wyckoff]

There are some old version of p4, compiled for cygwin, that
treat read-only files differently.

Normally, a file that is not open is read-only, meaning that
"test -w" on the file is false.  This works on unix, and it works
on windows using the NT version of p4.  The cygwin version
of p4, though, changes the permissions, but does not set the
windows read-only attribute, so "test -w" returns false.

Notice this oddity and make the tests work, even on cygiwn.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/lib-git-p4.sh               | 13 +++++++++++++
 t/t9807-git-p4-submit.sh      | 14 ++++++++++++--
 t/t9809-git-p4-client-view.sh |  4 ++--
 3 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
index fbd55ea..23d01fd 100644
--- a/t/lib-git-p4.sh
+++ b/t/lib-git-p4.sh
@@ -150,3 +150,16 @@ client_view() {
 		done
 	) | p4 client -i
 }
+
+is_cli_file_writeable() {
+	# cygwin version of p4 does not set read-only attr,
+	# will be marked 444 but -w is true
+	file="$1" &&
+	if test_have_prereq CYGWIN && p4 -V | grep -q CYGWIN
+	then
+		stat=$(stat --format=%a "$file") &&
+		test $stat = 644
+	else
+		test -w "$file"
+	fi
+}
diff --git a/t/t9807-git-p4-submit.sh b/t/t9807-git-p4-submit.sh
index 0ae048f..1fb7bc7 100755
--- a/t/t9807-git-p4-submit.sh
+++ b/t/t9807-git-p4-submit.sh
@@ -17,6 +17,16 @@ test_expect_success 'init depot' '
 	)
 '
 
+test_expect_failure 'is_cli_file_writeable function' '
+	(
+		cd "$cli" &&
+		echo a >a &&
+		is_cli_file_writeable a &&
+		! is_cli_file_writeable file1 &&
+		rm a
+	)
+'
+
 test_expect_success 'submit with no client dir' '
 	test_when_finished cleanup_git &&
 	git p4 clone --dest="$git" //depot &&
@@ -200,7 +210,7 @@ test_expect_success 'submit copy' '
 	(
 		cd "$cli" &&
 		test_path_is_file file5.ta &&
-		test ! -w file5.ta
+		! is_cli_file_writeable file5.ta
 	)
 '
 
@@ -219,7 +229,7 @@ test_expect_success 'submit rename' '
 		cd "$cli" &&
 		test_path_is_missing file6.t &&
 		test_path_is_file file6.ta &&
-		test ! -w file6.ta
+		! is_cli_file_writeable file6.ta
 	)
 '
 
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index fd8fa89..e0eb6b0 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -333,7 +333,7 @@ test_expect_success 'subdir clone, submit copy' '
 	(
 		cd "$cli" &&
 		test_path_is_file dir1/file11a &&
-		test ! -w dir1/file11a
+		! is_cli_file_writeable dir1/file11a
 	)
 '
 
@@ -353,7 +353,7 @@ test_expect_success 'subdir clone, submit rename' '
 		cd "$cli" &&
 		test_path_is_missing dir1/file13 &&
 		test_path_is_file dir1/file13a &&
-		test ! -w dir1/file13a
+		! is_cli_file_writeable dir1/file13a
 	)
 '
 
-- 
1.7.12.1.403.g28165e1

[PATCH 15/21] git p4 test: disable chmod test for cygwin

[Pete Wyckoff]

It does not notice chmod +x or -x; there is nothing
for this test to do.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 t/t9815-git-p4-submit-fail.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/t/t9815-git-p4-submit-fail.sh b/t/t9815-git-p4-submit-fail.sh
index d2b7b3d..2db1bf1 100755
--- a/t/t9815-git-p4-submit-fail.sh
+++ b/t/t9815-git-p4-submit-fail.sh
@@ -400,7 +400,9 @@ test_expect_success 'cleanup rename after submit cancel' '
 	)
 '
 
-test_expect_success 'cleanup chmod after submit cancel' '
+# chmods are not recognized in cygwin; git has nothing
+# to commit
+test_expect_success NOT_CYGWIN 'cleanup chmod after submit cancel' '
 	test_when_finished cleanup_git &&
 	git p4 clone --dest="$git" //depot &&
 	(
-- 
1.7.12.1.403.g28165e1

[PATCH 16/21] git p4: disable read-only attribute before deleting

[Pete Wyckoff]

On windows, p4 marks un-edited files as read-only.  Not only are
they read-only, but also they cannot be deleted.  Remove the
read-only attribute before deleting in both the copy and rename
cases.

This also happens in the RCS cleanup code, where a file is marked
to be deleted, but must first be edited to remove adjust the
keyword lines.  Make sure it is editable before patching.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/git-p4.py b/git-p4.py
index 5b2f73d..a6806bc 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -17,6 +17,7 @@ import time
 import platform
 import re
 import shutil
+import stat
 
 verbose = False
 
@@ -1163,6 +1164,9 @@ class P4Submit(Command, P4UserMap):
                     p4_edit(dest)
                     pureRenameCopy.discard(dest)
                     filesToChangeExecBit[dest] = diff['dst_mode']
+                if self.isWindows:
+                    # turn off read-only attribute
+                    os.chmod(dest, stat.S_IWRITE)
                 os.unlink(dest)
                 editedFiles.add(dest)
             elif modifier == "R":
@@ -1181,6 +1185,8 @@ class P4Submit(Command, P4UserMap):
                         p4_edit(dest)   # with move: already open, writable
                     filesToChangeExecBit[dest] = diff['dst_mode']
                 if not self.p4HasMoveCommand:
+                    if self.isWindows:
+                        os.chmod(dest, stat.S_IWRITE)
                     os.unlink(dest)
                     filesToDelete.add(src)
                 editedFiles.add(dest)
@@ -1221,6 +1227,10 @@ class P4Submit(Command, P4UserMap):
                 for file in kwfiles:
                     if verbose:
                         print "zapping %s with %s" % (line,pattern)
+                    # File is being deleted, so not open in p4.  Must
+                    # disable the read-only bit on windows.
+                    if self.isWindows and file not in editedFiles:
+                        os.chmod(file, stat.S_IWRITE)
                     self.patchRCSKeywords(file, kwfiles[file])
                     fixed_rcs_keywords = True
 
-- 
1.7.12.1.403.g28165e1

[PATCH 17/21] git p4: avoid shell when mapping users

[Pete Wyckoff]

The extra quoting and double-% are unneeded, just to work
around the shell.  Instead, avoid the shell indirection.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index a6806bc..a92d84f 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -982,7 +982,8 @@ class P4Submit(Command, P4UserMap):
     def p4UserForCommit(self,id):
         # Return the tuple (perforce user,git email) for a given git commit id
         self.getUserMapFromPerforceServer()
-        gitEmail = read_pipe("git log --max-count=1 --format='%%ae' %s" % id)
+        gitEmail = read_pipe(["git", "log", "--max-count=1",
+                              "--format=%ae", id])
         gitEmail = gitEmail.strip()
         if not self.emails.has_key(gitEmail):
             return (None,gitEmail)
-- 
1.7.12.1.403.g28165e1

[PATCH 18/21] git p4: avoid shell when invoking git rev-list

[Pete Wyckoff]

Invoke git rev-list directly, avoiding the shell, in
P4Submit and P4Sync.  The overhead of starting extra
processes is significant in cygwin; this speeds things
up on that platform.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index a92d84f..9c33af4 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -1538,7 +1538,7 @@ class P4Submit(Command, P4UserMap):
         self.check()
 
         commits = []
-        for line in read_pipe_lines("git rev-list --no-merges %s..%s" % (self.origin, self.master)):
+        for line in read_pipe_lines(["git", "rev-list", "--no-merges", "%s..%s" % (self.origin, self.master)]):
             commits.append(line.strip())
         commits.reverse()
 
@@ -2558,7 +2558,8 @@ class P4Sync(Command, P4UserMap):
 
     def searchParent(self, parent, branch, target):
         parentFound = False
-        for blob in read_pipe_lines(["git", "rev-list", "--reverse", "--no-merges", parent]):
+        for blob in read_pipe_lines(["git", "rev-list", "--reverse",
+                                     "--no-merges", parent]):
             blob = blob.strip()
             if len(read_pipe(["git", "diff-tree", blob, target])) == 0:
                 parentFound = True
-- 
1.7.12.1.403.g28165e1

[PATCH 19/21] git p4: avoid shell when invoking git config --get-all

[Pete Wyckoff]

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/git-p4.py b/git-p4.py
index 9c33af4..c0c738a 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -525,7 +525,8 @@ def gitConfig(key, args = None): # set args to "--bool", for instance
 
 def gitConfigList(key):
     if not _gitConfig.has_key(key):
-        _gitConfig[key] = read_pipe("git config --get-all %s" % key, ignore_error=True).strip().split(os.linesep)
+        s = read_pipe(["git", "config", "--get-all", key], ignore_error=True)
+        _gitConfig[key] = s.strip().split(os.linesep)
     return _gitConfig[key]
 
 def p4BranchesInGit(branchesAreInRemotes = True):
-- 
1.7.12.1.403.g28165e1

[PATCH 20/21] git p4: avoid shell when calling git config

[Pete Wyckoff]

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index c0c738a..007ef6b 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -514,13 +514,16 @@ def gitBranchExists(branch):
     return proc.wait() == 0;
 
 _gitConfig = {}
-def gitConfig(key, args = None): # set args to "--bool", for instance
+
+def gitConfig(key, args=None): # set args to "--bool", for instance
     if not _gitConfig.has_key(key):
-        argsFilter = ""
-        if args != None:
-            argsFilter = "%s " % args
-        cmd = "git config %s%s" % (argsFilter, key)
-        _gitConfig[key] = read_pipe(cmd, ignore_error=True).strip()
+        cmd = [ "git", "config" ]
+        if args:
+            assert(args == "--bool")
+            cmd.append(args)
+        cmd.append(key)
+        s = read_pipe(cmd, ignore_error=True)
+        _gitConfig[key] = s.strip()
     return _gitConfig[key]
 
 def gitConfigList(key):
-- 
1.7.12.1.403.g28165e1

[PATCH 21/21] git p4: introduce gitConfigBool

[Pete Wyckoff]

Make the intent of "--bool" more obvious by returning a direct True
or False value.  Convert a couple non-bool users with obvious bool
intent.

Signed-off-by: Pete Wyckoff <pw@padd.com>
---
 git-p4.py | 45 ++++++++++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 19 deletions(-)

diff --git a/git-p4.py b/git-p4.py
index 007ef6b..524df12 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -515,17 +515,25 @@ def gitBranchExists(branch):
 
 _gitConfig = {}
 
-def gitConfig(key, args=None): # set args to "--bool", for instance
+def gitConfig(key):
     if not _gitConfig.has_key(key):
-        cmd = [ "git", "config" ]
-        if args:
-            assert(args == "--bool")
-            cmd.append(args)
-        cmd.append(key)
+        cmd = [ "git", "config", key ]
         s = read_pipe(cmd, ignore_error=True)
         _gitConfig[key] = s.strip()
     return _gitConfig[key]
 
+def gitConfigBool(key):
+    """Return a bool, using git config --bool.  It is True only if the
+       variable is set to true, and False if set to false or not present
+       in the config."""
+
+    if not _gitConfig.has_key(key):
+        cmd = [ "git", "config", "--bool", key ]
+        s = read_pipe(cmd, ignore_error=True)
+        v = s.strip()
+        _gitConfig[key] = v == "true"
+    return _gitConfig[key]
+
 def gitConfigList(key):
     if not _gitConfig.has_key(key):
         s = read_pipe(["git", "config", "--get-all", key], ignore_error=True)
@@ -656,8 +664,7 @@ def p4PathStartsWith(path, prefix):
     #
     # we may or may not have a problem. If you have core.ignorecase=true,
     # we treat DirA and dira as the same directory
-    ignorecase = gitConfig("core.ignorecase", "--bool") == "true"
-    if ignorecase:
+    if gitConfigBool("core.ignorecase"):
         return path.lower().startswith(prefix.lower())
     return path.startswith(prefix)
 
@@ -892,7 +899,7 @@ class P4Submit(Command, P4UserMap):
         self.usage += " [name of git branch to submit into perforce depot]"
         self.origin = ""
         self.detectRenames = False
-        self.preserveUser = gitConfig("git-p4.preserveUser").lower() == "true"
+        self.preserveUser = gitConfigBool("git-p4.preserveUser")
         self.dry_run = False
         self.prepare_p4_only = False
         self.conflict_behavior = None
@@ -1000,7 +1007,7 @@ class P4Submit(Command, P4UserMap):
             (user,email) = self.p4UserForCommit(id)
             if not user:
                 msg = "Cannot find p4 user for email %s in commit %s." % (email, id)
-                if gitConfig('git-p4.allowMissingP4Users').lower() == "true":
+                if gitConfigBool("git-p4.allowMissingP4Users"):
                     print "%s" % msg
                 else:
                     die("Error: %s\nSet git-p4.allowMissingP4Users to true to allow this." % msg)
@@ -1095,7 +1102,7 @@ class P4Submit(Command, P4UserMap):
            message.  Return true if okay to continue with the submit."""
 
         # if configured to skip the editing part, just submit
-        if gitConfig("git-p4.skipSubmitEdit") == "true":
+        if gitConfigBool("git-p4.skipSubmitEdit"):
             return True
 
         # look at the modification time, to check later if the user saved
@@ -1111,7 +1118,7 @@ class P4Submit(Command, P4UserMap):
 
         # If the file was not saved, prompt to see if this patch should
         # be skipped.  But skip this verification step if configured so.
-        if gitConfig("git-p4.skipSubmitEditCheck") == "true":
+        if gitConfigBool("git-p4.skipSubmitEditCheck"):
             return True
 
         # modification time updated means user saved the file
@@ -1211,7 +1218,7 @@ class P4Submit(Command, P4UserMap):
 
             # Patch failed, maybe it's just RCS keyword woes. Look through
             # the patch to see if that's possible.
-            if gitConfig("git-p4.attemptRCSCleanup","--bool") == "true":
+            if gitConfigBool("git-p4.attemptRCSCleanup"):
                 file = None
                 pattern = None
                 kwfiles = {}
@@ -1506,7 +1513,7 @@ class P4Submit(Command, P4UserMap):
             sys.exit(128)
 
         self.useClientSpec = False
-        if gitConfig("git-p4.useclientspec", "--bool") == "true":
+        if gitConfigBool("git-p4.useclientspec"):
             self.useClientSpec = True
         if self.useClientSpec:
             self.clientSpecDirs = getClientSpec()
@@ -1546,7 +1553,7 @@ class P4Submit(Command, P4UserMap):
             commits.append(line.strip())
         commits.reverse()
 
-        if self.preserveUser or (gitConfig("git-p4.skipUserNameCheck") == "true"):
+        if self.preserveUser or gitConfigBool("git-p4.skipUserNameCheck"):
             self.checkAuthorship = False
         else:
             self.checkAuthorship = True
@@ -1582,7 +1589,7 @@ class P4Submit(Command, P4UserMap):
         else:
             self.diffOpts += " -C%s" % detectCopies
 
-        if gitConfig("git-p4.detectCopiesHarder", "--bool") == "true":
+        if gitConfigBool("git-p4.detectCopiesHarder"):
             self.diffOpts += " --find-copies-harder"
 
         #
@@ -1664,7 +1671,7 @@ class P4Submit(Command, P4UserMap):
                                            "--format=format:%h %s",  c])
                 print "You will have to do 'git p4 sync' and rebase."
 
-        if gitConfig("git-p4.exportLabels", "--bool") == "true":
+        if gitConfigBool("git-p4.exportLabels"):
             self.exportLabels = True
 
         if self.exportLabels:
@@ -2757,7 +2764,7 @@ class P4Sync(Command, P4UserMap):
             # will use this after clone to set the variable
             self.useClientSpec_from_options = True
         else:
-            if gitConfig("git-p4.useclientspec", "--bool") == "true":
+            if gitConfigBool("git-p4.useclientspec"):
                 self.useClientSpec = True
         if self.useClientSpec:
             self.clientSpecDirs = getClientSpec()
@@ -2954,7 +2961,7 @@ class P4Sync(Command, P4UserMap):
                             sys.stdout.write("%s " % b)
                         sys.stdout.write("\n")
 
-        if gitConfig("git-p4.importLabels", "--bool") == "true":
+        if gitConfigBool("git-p4.importLabels"):
             self.importLabels = True
 
         if self.importLabels:
-- 
1.7.12.1.403.g28165e1

Re: [PATCH 03/21] git p4: generate better error message for bad depot path

[Junio C Hamano]

Pete Wyckoff <pw@padd.com> writes:

> Depot paths must start with //.  Exit with a better explanation
> when a bad depot path is supplied.
>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
>  git-p4.py               | 1 +
>  t/t9800-git-p4-basic.sh | 5 +++++
>  2 files changed, 6 insertions(+)
>
> diff --git a/git-p4.py b/git-p4.py
> index 97699ef..eef5c94 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -3035,6 +3035,7 @@ class P4Clone(P4Sync):
>          self.cloneExclude = ["/"+p for p in self.cloneExclude]
>          for p in depotPaths:
>              if not p.startswith("//"):
> +                sys.stderr.write('Depot paths must start with "//": %s\n' % p)
>                  return False
>  
>          if not self.cloneDestination:
> diff --git a/t/t9800-git-p4-basic.sh b/t/t9800-git-p4-basic.sh
> index b7ad716..c5f4c88 100755
> --- a/t/t9800-git-p4-basic.sh
> +++ b/t/t9800-git-p4-basic.sh
> @@ -30,6 +30,11 @@ test_expect_success 'basic git p4 clone' '
>  	)
>  '
>  
> +test_expect_success 'depot typo error' '
> +	test_must_fail git p4 clone --dest="$git" /depot 2>errs &&
> +	grep -q "Depot paths must start with" errs
> +'

Use of "grep -q" does not help ordinary testers, as the output will
not be shown when the tests are run normally.

>  test_expect_success 'git p4 clone @all' '
>  	git p4 clone --dest="$git" //depot@all &&
>  	test_when_finished cleanup_git &&

Re: [PATCH 04/21] git p4: fix error message when "describe -s" fails

[Junio C Hamano]

Pete Wyckoff <pw@padd.com> writes:

> The output was a bit nonsensical, including a bare %d.  Fix it
> to make it easier to understand.
>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
>  git-p4.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/git-p4.py b/git-p4.py
> index eef5c94..d7ee4b4 100755
> --- a/git-p4.py
> +++ b/git-p4.py
> @@ -2679,7 +2679,8 @@ class P4Sync(Command, P4UserMap):
>              if r.has_key('time'):
>                  newestTime = int(r['time'])
>          if newestTime is None:
> -            die("\"describe -s\" on newest change %d did not give a time")
> +            die("Output from \"describe -s\" on newest change %d did not give a time" %
> +                newestRevision)

Shouldn't it say "p4 describe -s"?

>          details["time"] = newestTime
>  
>          self.updateOptionDict(details)

Re: [PATCH 05/21] git p4 test: use client_view to build the initial client

[Junio C Hamano]

Pete Wyckoff <pw@padd.com> writes:

> Simplify the code a bit by using an existing function.
>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
>  t/lib-git-p4.sh | 11 ++---------
>  1 file changed, 2 insertions(+), 9 deletions(-)
>
> diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
> index 7061dce..890ee60 100644
> --- a/t/lib-git-p4.sh
> +++ b/t/lib-git-p4.sh
> @@ -74,15 +74,8 @@ start_p4d() {
>  	fi
>  
>  	# build a client
> -	(
> -		cd "$cli" &&
> -		p4 client -i <<-EOF
> -		Client: client
> -		Description: client
> -		Root: $cli
> -		View: //depot/... //client/...
> -		EOF
> -	)
> +	client_view "//depot/... //client/..." &&
> +
>  	return 0
>  }

Assuming that writing //depot/... //client/... on the next line
indented by a tab is equivalent to writing it on View: line (which I
think it is), this looks like an obviously good reuse of the code.

I have to wonder if the use of printf in client_view implementation
should be tighted up, though.

diff --git i/t/lib-git-p4.sh w/t/lib-git-p4.sh
index 7061dce..4e58289 100644
--- i/t/lib-git-p4.sh
+++ w/t/lib-git-p4.sh
@@ -128,8 +128,6 @@ client_view() {
 		Root: $cli
 		View:
 		EOF
-		for arg ; do
-			printf "\t$arg\n"
-		done
+		printf "\t%s\n" "$@"
 	) | p4 client -i
 }

Re: [PATCH 06/21] git p4 test: use client_view in t9806

[Junio C Hamano]

Pete Wyckoff <pw@padd.com> writes:

> Use the standard client_view function from lib-git-p4.sh
> instead of building one by hand.  This requires a bit of
> rework, using the current value of $P4CLIENT for the client
> name.  It also reorganizes the test to isolate changes to
> $P4CLIENT and $cli in a subshell.
>
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
>  t/lib-git-p4.sh           |  4 ++--
>  t/t9806-git-p4-options.sh | 50 ++++++++++++++++++++++-------------------------
>  2 files changed, 25 insertions(+), 29 deletions(-)
>
> diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
> index 890ee60..d558dd0 100644
> --- a/t/lib-git-p4.sh
> +++ b/t/lib-git-p4.sh
> @@ -116,8 +116,8 @@ marshal_dump() {
>  client_view() {
>  	(
>  		cat <<-EOF &&
> -		Client: client
> -		Description: client
> +		Client: $P4CLIENT
> +		Description: $P4CLIENT
>  		Root: $cli
>  		View:
>  		EOF
> diff --git a/t/t9806-git-p4-options.sh b/t/t9806-git-p4-options.sh
> index fa40cc8..37ca30a 100755
> --- a/t/t9806-git-p4-options.sh
> +++ b/t/t9806-git-p4-options.sh
> @@ -126,37 +126,33 @@ test_expect_success 'clone --use-client-spec' '
>  		exec >/dev/null &&
>  		test_must_fail git p4 clone --dest="$git" --use-client-spec
>  	) &&
> -	cli2=$(test-path-utils real_path "$TRASH_DIRECTORY/cli2") &&
> +	# build a different client
> +	cli2="$TRASH_DIRECTORY/cli2" &&
>  	mkdir -p "$cli2" &&
>  	test_when_finished "rmdir \"$cli2\"" &&
>  	test_when_finished cleanup_git &&
> ...
> -	# same thing again, this time with variable instead of option
>  	(
> ...
> +		# group P4CLIENT and cli changes in a sub-shell
> +		P4CLIENT=client2 &&
> +		cli="$cli2" &&
> +		client_view "//depot/sub/... //client2/bus/..." &&
> +		git p4 clone --dest="$git" --use-client-spec //depot/... &&
> +		(
> +			cd "$git" &&
> +			test_path_is_file bus/dir/f4 &&
> +			test_path_is_missing file1
> +		) &&
> +		cleanup_git &&

Hmm, the use of "test-path-utils real_path" to form cli2 in the
original was not necessary at all?

> +		# same thing again, this time with variable instead of option
> +		(
> +			cd "$git" &&
> +			git init &&
> +			git config git-p4.useClientSpec true &&
> +			git p4 sync //depot/... &&
> +			git checkout -b master p4/master &&
> +			test_path_is_file bus/dir/f4 &&
> +			test_path_is_missing file1
> +		)

Do you need a separate sub-shell inside a sub-shell we are already
in that you called client_view in?

>  	)
>  '

Re: [PATCH 00/21] git p4: work on cygwin

[Junio C Hamano]

Pete Wyckoff <pw@padd.com> writes:

> This series fixes problems in git-p4, and its tests, so that
> git-p4 works on the cygwin platform.
>
> See the wiki for info on how to get started on cygwin:
>
>     https://git.wiki.kernel.org/index.php/GitP4
>
> Testing by people who use cygwin would be appreciated.  It would
> be good to support cygwin more regularly.  Anyone who had time
> to contribute to testing on cygwin, and reporting problems, would
> be welcome.
>
> There's more work requried to support msysgit.  Those patches
> are not in good enough shape to ship out yet, but a lot of what
> is in this series is required for msysgit too.
>
> These patches:
>
>     - fix bugs in git-p4 related to issues found on cygwin
>     - cleanup some ugly code in git-p4 observed in error paths while
>       getting tests to work on cygwin
>     - simplify and refactor code and tests to make cygwin changes easier
>     - handle newline and path issues for cygwin platform
>     - speed up some aspects of git-p4 by removing extra shell invocations
>
> Pete Wyckoff (21):
>   git p4: temp branch name should use / even on windows
>   git p4: remove unused imports
>   git p4: generate better error message for bad depot path
>   git p4: fix error message when "describe -s" fails
>   git p4 test: use client_view to build the initial client
>   git p4 test: use client_view in t9806
>   git p4 test: start p4d inside its db dir
>   git p4 test: translate windows paths for cygwin
>   git p4: remove unreachable windows \r\n conversion code
>   git p4: scrub crlf for utf16 files on windows
>   git p4 test: newline handling
>   git p4 test: use LineEnd unix in windows tests too
>   git p4 test: avoid wildcard * in windows
>   git p4: cygwin p4 client does not mark read-only
>   git p4 test: disable chmod test for cygwin
>   git p4: disable read-only attribute before deleting
>   git p4: avoid shell when mapping users
>   git p4: avoid shell when invoking git rev-list
>   git p4: avoid shell when invoking git config --get-all
>   git p4: avoid shell when calling git config
>   git p4: introduce gitConfigBool

Very nicely done.  I was impressed how easy to understand what the
problem each patch attempts to solve is and how it should be solved
only from the description and the changes apparently matched the
description.

I wish patches from everybody looked like these.

Re: [PATCH 15/21] git p4 test: disable chmod test for cygwin

[Johannes Sixt]

Am 28.09.2012 14:04, schrieb Pete Wyckoff:
> It does not notice chmod +x or -x; there is nothing
> for this test to do.
> 
> Signed-off-by: Pete Wyckoff <pw@padd.com>
> ---
>  t/t9815-git-p4-submit-fail.sh | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/t/t9815-git-p4-submit-fail.sh b/t/t9815-git-p4-submit-fail.sh
> index d2b7b3d..2db1bf1 100755
> --- a/t/t9815-git-p4-submit-fail.sh
> +++ b/t/t9815-git-p4-submit-fail.sh
> @@ -400,7 +400,9 @@ test_expect_success 'cleanup rename after submit cancel' '
>  	)
>  '
>  
> -test_expect_success 'cleanup chmod after submit cancel' '
> +# chmods are not recognized in cygwin; git has nothing
> +# to commit
> +test_expect_success NOT_CYGWIN 'cleanup chmod after submit cancel' '
>  	test_when_finished cleanup_git &&
>  	git p4 clone --dest="$git" //depot &&
>  	(
> 

In the git part, you could use test_chmod to change the executable bit.
But if you cannot test it in the p4 part later on, it is probably not
worth it.

-- Hannes

Re: [PATCH 06/21] git p4 test: use client_view in t9806

[Pete Wyckoff]

Yes, this really is four months later.  Somehow I forgot all
about this series.

gitster@pobox.com wrote on Fri, 28 Sep 2012 12:11 -0700:
> Pete Wyckoff <pw@padd.com> writes:
> 
> > Use the standard client_view function from lib-git-p4.sh
> > instead of building one by hand.  This requires a bit of
> > rework, using the current value of $P4CLIENT for the client
> > name.  It also reorganizes the test to isolate changes to
> > $P4CLIENT and $cli in a subshell.
> >
> > Signed-off-by: Pete Wyckoff <pw@padd.com>
> > ---
> >  t/lib-git-p4.sh           |  4 ++--
> >  t/t9806-git-p4-options.sh | 50 ++++++++++++++++++++++-------------------------
> >  2 files changed, 25 insertions(+), 29 deletions(-)
> >
> > diff --git a/t/lib-git-p4.sh b/t/lib-git-p4.sh
> > index 890ee60..d558dd0 100644
> > --- a/t/lib-git-p4.sh
> > +++ b/t/lib-git-p4.sh
> > @@ -116,8 +116,8 @@ marshal_dump() {
> >  client_view() {
> >  	(
> >  		cat <<-EOF &&
> > -		Client: client
> > -		Description: client
> > +		Client: $P4CLIENT
> > +		Description: $P4CLIENT
> >  		Root: $cli
> >  		View:
> >  		EOF
> > diff --git a/t/t9806-git-p4-options.sh b/t/t9806-git-p4-options.sh
> > index fa40cc8..37ca30a 100755
> > --- a/t/t9806-git-p4-options.sh
> > +++ b/t/t9806-git-p4-options.sh
> > @@ -126,37 +126,33 @@ test_expect_success 'clone --use-client-spec' '
> >  		exec >/dev/null &&
> >  		test_must_fail git p4 clone --dest="$git" --use-client-spec
> >  	) &&
> > -	cli2=$(test-path-utils real_path "$TRASH_DIRECTORY/cli2") &&
> > +	# build a different client
> > +	cli2="$TRASH_DIRECTORY/cli2" &&
> >  	mkdir -p "$cli2" &&
> >  	test_when_finished "rmdir \"$cli2\"" &&
> >  	test_when_finished cleanup_git &&
> > ...
> > -	# same thing again, this time with variable instead of option
> >  	(
> > ...
> > +		# group P4CLIENT and cli changes in a sub-shell
> > +		P4CLIENT=client2 &&
> > +		cli="$cli2" &&
> > +		client_view "//depot/sub/... //client2/bus/..." &&
> > +		git p4 clone --dest="$git" --use-client-spec //depot/... &&
> > +		(
> > +			cd "$git" &&
> > +			test_path_is_file bus/dir/f4 &&
> > +			test_path_is_missing file1
> > +		) &&
> > +		cleanup_git &&
> 
> Hmm, the use of "test-path-utils real_path" to form cli2 in the
> original was not necessary at all?

Thanks, I will make this removal more explicit, putting it in
with 8/21 where it belongs, with explanation.

> > +		# same thing again, this time with variable instead of option
> > +		(
> > +			cd "$git" &&
> > +			git init &&
> > +			git config git-p4.useClientSpec true &&
> > +			git p4 sync //depot/... &&
> > +			git checkout -b master p4/master &&
> > +			test_path_is_file bus/dir/f4 &&
> > +			test_path_is_missing file1
> > +		)
> 
> Do you need a separate sub-shell inside a sub-shell we are already
> in that you called client_view in?
> 
> >  	)
> >  '

The first subshell is to hide P4CLIENT and cli variable changes
from the rest of the tests.

The second is to keep the "cd $git" from changing behavior of the
following "cleanup_git" call.  That does "rm -rf $git" which
would fail on some file systems if cwd is still in there.  With
just one subshell it would look like:

	(
		P4CLIENT=client2 &&
		git p4 clone .. &&
		cd "$git" &&
		... do test
		cd "$TRASH_DIRECTORY" &&
		cleanup_git &&

		cd "$git" &&
		... more test
	)

It's a bit easier to understand with an extra level of shell,
and sticks to the pattern used in the rest of the t98*.

		-- Pete