From: Lars Schneider <larsxschneider@gmail.com>
To: git@vger.kernel.org
Cc: luke@diamand.org, gitster@pobox.com
Subject: [PATCH v2] git-p4: fix git-p4.pathEncoding for removed files
Date: Thu, 9 Feb 2017 16:06:56 +0100 [thread overview]
Message-ID: <20170209150656.9070-1-larsxschneider@gmail.com> (raw)
In-Reply-To: <CAE5ih7-=bD_ZoL5pFYfD2Qvy-XE24V_cgge0XoAvuoTK02EDfg@mail.gmail.com>
In a9e38359e3 we taught git-p4 a way to re-encode path names from what
was used in Perforce to UTF-8. This path re-encoding worked properly for
"added" paths. "Removed" paths were not re-encoded and therefore
different from the "added" paths. Consequently, these files were not
removed in a git-p4 cloned Git repository because the path names did not
match.
Fix this by moving the re-encoding to a place that affects "added" and
"removed" paths. Add a test to demonstrate the issue.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
Hi,
unfortunately, I missed to send this v2. I agree with Luke's review and
I moved the re-encode of the path name to the `streamOneP4File` and
`streamOneP4Deletion` explicitly.
Discussion:
http://public-inbox.org/git/CAE5ih7-=bD_ZoL5pFYfD2Qvy-XE24V_cgge0XoAvuoTK02EDfg@mail.gmail.com/
Thanks,
Lars
Notes:
Base Commit: 454cb6bd52 (v2.11.0)
Diff on Web: https://github.com/larsxschneider/git/commit/75ed3e92e2
Checkout: git fetch https://github.com/larsxschneider/git git-p4/fix-path-encoding-v2 && git checkout 75ed3e92e2
Interdiff (v1..v2):
diff --git a/git-p4.py b/git-p4.py
index 8f311cb4e8..dac8b4955d 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2366,15 +2366,6 @@ class P4Sync(Command, P4UserMap):
break
path = wildcard_decode(path)
- try:
- path.decode('ascii')
- except:
- encoding = 'utf8'
- if gitConfig('git-p4.pathEncoding'):
- encoding = gitConfig('git-p4.pathEncoding')
- path = path.decode(encoding, 'replace').encode('utf8', 'replace')
- if self.verbose:
- print 'Path with non-ASCII characters detected. Used %s to encode: %s ' % (encoding, path)
return path
def splitFilesIntoBranches(self, commit):
@@ -2427,11 +2418,24 @@ class P4Sync(Command, P4UserMap):
self.gitStream.write(d)
self.gitStream.write('\n')
+ def encodeWithUTF8(self, path):
+ try:
+ path.decode('ascii')
+ except:
+ encoding = 'utf8'
+ if gitConfig('git-p4.pathEncoding'):
+ encoding = gitConfig('git-p4.pathEncoding')
+ path = path.decode(encoding, 'replace').encode('utf8', 'replace')
+ if self.verbose:
+ print 'Path with non-ASCII characters detected. Used %s to encode: %s ' % (encoding, path)
+ return path
+
# output one file from the P4 stream
# - helper for streamP4Files
def streamOneP4File(self, file, contents):
relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
+ relPath = self.encodeWithUTF8(relPath)
if verbose:
size = int(self.stream_file['fileSize'])
sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
@@ -2511,6 +2515,7 @@ class P4Sync(Command, P4UserMap):
def streamOneP4Deletion(self, file):
relPath = self.stripRepoPath(file['path'], self.branchPrefixes)
+ relPath = self.encodeWithUTF8(relPath)
if verbose:
sys.stdout.write("delete %s\n" % relPath)
sys.stdout.flush()
git-p4.py | 24 ++++++++++++++----------
t/t9822-git-p4-path-encoding.sh | 16 ++++++++++++++++
2 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/git-p4.py b/git-p4.py
index fd5ca52462..dac8b4955d 100755
--- a/git-p4.py
+++ b/git-p4.py
@@ -2418,11 +2418,24 @@ class P4Sync(Command, P4UserMap):
self.gitStream.write(d)
self.gitStream.write('\n')
+ def encodeWithUTF8(self, path):
+ try:
+ path.decode('ascii')
+ except:
+ encoding = 'utf8'
+ if gitConfig('git-p4.pathEncoding'):
+ encoding = gitConfig('git-p4.pathEncoding')
+ path = path.decode(encoding, 'replace').encode('utf8', 'replace')
+ if self.verbose:
+ print 'Path with non-ASCII characters detected. Used %s to encode: %s ' % (encoding, path)
+ return path
+
# output one file from the P4 stream
# - helper for streamP4Files
def streamOneP4File(self, file, contents):
relPath = self.stripRepoPath(file['depotFile'], self.branchPrefixes)
+ relPath = self.encodeWithUTF8(relPath)
if verbose:
size = int(self.stream_file['fileSize'])
sys.stdout.write('\r%s --> %s (%i MB)\n' % (file['depotFile'], relPath, size/1024/1024))
@@ -2495,16 +2508,6 @@ class P4Sync(Command, P4UserMap):
text = regexp.sub(r'$\1$', text)
contents = [ text ]
- try:
- relPath.decode('ascii')
- except:
- encoding = 'utf8'
- if gitConfig('git-p4.pathEncoding'):
- encoding = gitConfig('git-p4.pathEncoding')
- relPath = relPath.decode(encoding, 'replace').encode('utf8', 'replace')
- if self.verbose:
- print 'Path with non-ASCII characters detected. Used %s to encode: %s ' % (encoding, relPath)
-
if self.largeFileSystem:
(git_mode, contents) = self.largeFileSystem.processContent(git_mode, relPath, contents)
@@ -2512,6 +2515,7 @@ class P4Sync(Command, P4UserMap):
def streamOneP4Deletion(self, file):
relPath = self.stripRepoPath(file['path'], self.branchPrefixes)
+ relPath = self.encodeWithUTF8(relPath)
if verbose:
sys.stdout.write("delete %s\n" % relPath)
sys.stdout.flush()
diff --git a/t/t9822-git-p4-path-encoding.sh b/t/t9822-git-p4-path-encoding.sh
index 7b83e696a9..c78477c19b 100755
--- a/t/t9822-git-p4-path-encoding.sh
+++ b/t/t9822-git-p4-path-encoding.sh
@@ -51,6 +51,22 @@ test_expect_success 'Clone repo containing iso8859-1 encoded paths with git-p4.p
)
'
+test_expect_success 'Delete iso8859-1 encoded paths and clone' '
+ (
+ cd "$cli" &&
+ ISO8859="$(printf "$ISO8859_ESCAPED")" &&
+ p4 delete "$ISO8859" &&
+ p4 submit -d "remove file"
+ ) &&
+ git p4 clone --destination="$git" //depot@all &&
+ test_when_finished cleanup_git &&
+ (
+ cd "$git" &&
+ git -c core.quotepath=false ls-files >actual &&
+ test_must_be_empty actual
+ )
+'
+
test_expect_success 'kill p4d' '
kill_p4d
'
base-commit: 454cb6bd52a4de614a3633e4f547af03d5c3b640
--
2.11.0
next prev parent reply other threads:[~2017-02-09 15:07 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-18 17:51 [PATCH v1] git-p4: fix git-p4.pathEncoding for removed files larsxschneider
2016-12-19 21:29 ` Junio C Hamano
2016-12-20 11:01 ` Luke Diamand
2016-12-22 21:23 ` Junio C Hamano
2017-02-09 15:06 ` Lars Schneider [this message]
2017-02-09 23:39 ` [PATCH v2] " Junio C Hamano
2017-02-10 22:05 ` Luke Diamand
2017-02-10 22:32 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170209150656.9070-1-larsxschneider@gmail.com \
--to=larsxschneider@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=luke@diamand.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).