list mirror (unofficial, one of many)
 help / color / Atom feed
From: Yang Zhao <>
Cc: Yang Zhao <>
Subject: [RFC PATCH 1/4] git-p4: decode response from p4 to str for python3
Date: Wed, 27 Nov 2019 17:28:04 -0800
Message-ID: <> (raw)
In-Reply-To: <>

The marshalled dict in the response given on STDOUT by p4 uses `str` for
keys and string values. When run using python3, these values are
deserialized as `bytes`, leading to a whole host of problems as the rest
of the code assumes `str` is used throughout.

An exception is made for the `data` field as it may contain arbitrary
binary data that is not text, as well as `depotFile` which may contain
text encoded with something other than ASCII or UTF-8.

Signed-off-by: Yang Zhao <>
--- | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/ b/
index 60c73b6a37..ead9d816e1 100755
--- a/
+++ b/
@@ -36,6 +36,7 @@
     unicode = str
     bytes = bytes
     basestring = (str,bytes)
+    use_encoded_streams = True
     # 'unicode' exists, must be Python 2
     str = str
@@ -643,6 +644,15 @@ def p4CmdList(cmd, stdin=None, stdin_mode='w+b', cb=None, skip_info=False,
         while True:
             entry = marshal.load(p4.stdout)
+            if use_encoded_streams:
+                # Decode unmarshalled dict to use str keys and values, except for:
+                #   - `data` which may contain arbitrary binary data
+                #   - `depotFile` which may contain non-UTF8 encoded text
+                decoded_entry = {}
+                for key, value in entry.items():
+                    key = key.decode()
+                    decoded_entry[key] = value.decode() if not (key in ['data', 'depotFile'] or isinstance(value, str)) else value
+                entry = decoded_entry
             if skip_info:
                 if 'code' in entry and entry['code'] == 'info':

  reply index

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-28  1:28 [RFC PATCH 0/4] git-p4: python 3 compatability Yang Zhao
2019-11-28  1:28 ` Yang Zhao [this message]
2019-11-28  1:28 ` [RFC PATCH 2/4] git-p4: properly encode/decode communication with git for python 3 Yang Zhao
2019-11-28  1:28 ` [RFC PATCH 3/4] git-p4: open .gitp4-usercache.txt in text mode Yang Zhao
2019-11-28  1:28 ` [RFC PATCH 4/4] git-p4: use utf-8 encoding for file paths throughout Yang Zhao
2019-11-28  2:57   ` Elijah Newren
2019-11-28 12:54 ` [RFC PATCH 0/4] git-p4: python 3 compatability Johannes Schindelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:

 note: .onion URLs require Tor:

AGPL code for this site: git clone