git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

[Ivan Selin]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

Hello!

If I create a perforce client from java using p4java, it gets created
with an extra key "ViewDepotType" in client definition. When I try to
do `git p4 sync --use-client-spec`, git-p4 dies with message like
"Expected view key View1 missing" — because it assumes that all keys
starting with "View" are "View0", "View1" and so on.

How to reproduce:
1) Create perforce repository;
2) Create a perforce client (let's name it "my-client") in said
perforce repository using p4java; add one view mapping to the client
on creation;
3) Run `P4CLIENT=my-client git p4 sync --use-client-spec`;
4) git p4 finishes with error "Expected view key View1 missing".

Attaching hexdumped/unmarshalled examples of "regular" client
definition and client created with p4java. Note that p4java's version
has "ViewDepotType" key and regular client does not. Also,
"ViewDepotType" key is not showing up in text output of `p4 client
-o`, only in binary format (`p4 -G client -o`). And I'm attaching a
patch that solved the issue for me.

Is that information enough or should I add anything else? I'm new to
git community, but willing to help.

Regards,
Ivan.

[-- Attachment #2: p4-client-regular.txt --]
[-- Type: text/plain, Size: 2581 bytes --]

  $ p4 -G client -o c-1 | hexdump -C
00000000  7b 73 04 00 00 00 63 6f  64 65 73 04 00 00 00 73  |{s....codes....s|
00000010  74 61 74 73 06 00 00 00  43 6c 69 65 6e 74 73 03  |tats....Clients.|
00000020  00 00 00 63 2d 31 73 06  00 00 00 55 70 64 61 74  |...c-1s....Updat|
00000030  65 73 13 00 00 00 32 30  32 30 2f 30 31 2f 30 35  |es....2020/01/05|
00000040  20 31 36 3a 34 35 3a 33  36 73 06 00 00 00 41 63  | 16:45:36s....Ac|
00000050  63 65 73 73 73 13 00 00  00 32 30 32 30 2f 30 31  |cesss....2020/01|
00000060  2f 30 35 20 31 36 3a 34  35 3a 33 36 73 04 00 00  |/05 16:45:36s...|
00000070  00 52 6f 6f 74 73 3e 00  00 00 2f 68 6f 6d 65 2f  |.Roots>.../home/|
00000080  64 69 72 65 63 74 6f 72  79 2f 64 65 76 2f 66 72  |directory/dev/fr|
00000090  65 65 2f 67 69 74 2d 70  34 2f 70 34 2d 73 61 6d  |ee/git-p4/p4-sam|
000000a0  70 6c 65 2d 72 65 70 6f  2f 64 69 72 65 63 74 2d  |ple-repo/direct-|
000000b0  63 6d 64 2d 74 65 73 74  73 07 00 00 00 4f 70 74  |cmd-tests....Opt|
000000c0  69 6f 6e 73 73 3a 00 00  00 6e 6f 61 6c 6c 77 72  |ionss:...noallwr|
000000d0  69 74 65 20 6e 6f 63 6c  6f 62 62 65 72 20 6e 6f  |ite noclobber no|
000000e0  63 6f 6d 70 72 65 73 73  20 75 6e 6c 6f 63 6b 65  |compress unlocke|
000000f0  64 20 6e 6f 6d 6f 64 74  69 6d 65 20 6e 6f 72 6d  |d nomodtime norm|
00000100  64 69 72 73 0d 00 00 00  53 75 62 6d 69 74 4f 70  |dirs....SubmitOp|
00000110  74 69 6f 6e 73 73 0f 00  00 00 73 75 62 6d 69 74  |tionss....submit|
00000120  75 6e 63 68 61 6e 67 65  64 73 07 00 00 00 4c 69  |unchangeds....Li|
00000130  6e 65 45 6e 64 73 05 00  00 00 6c 6f 63 61 6c 73  |neEnds....locals|
00000140  05 00 00 00 56 69 65 77  30 73 15 00 00 00 2f 2f  |....View0s....//|
00000150  64 65 70 6f 74 2f 2e 2e  2e 20 2f 2f 63 2d 31 2f  |depot/... //c-1/|
00000160  2e 2e 2e 73 04 00 00 00  54 79 70 65 73 09 00 00  |...s....Types...|
00000170  00 77 72 69 74 65 61 62  6c 65 73 06 00 00 00 42  |.writeables....B|
00000180  61 63 6b 75 70 73 06 00  00 00 65 6e 61 62 6c 65  |ackups....enable|
00000190  30                                                |0|
00000191

  $ p4 -G client -o c-1 | python -c 'import marshal, pprint, sys; pprint.pprint(marshal.load(sys.stdin))'
{'Access': '2020/01/05 16:45:36',
 'Backup': 'enable',
 'Client': 'c-1',
 'LineEnd': 'local',
 'Options': 'noallwrite noclobber nocompress unlocked nomodtime normdir',
 'Root': '/home/directory/dev/free/git-p4/p4-sample-repo/direct-cmd-test',
 'SubmitOptions': 'submitunchanged',
 'Type': 'writeable',
 'Update': '2020/01/05 16:45:36',
 'View0': '//depot/... //c-1/...',
 'code': 'stat'}


[-- Attachment #3: git-p4-view-keys.patch --]
[-- Type: text/x-patch, Size: 557 bytes --]

 git-p4.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git git-p4.py git-p4.py
index ca0a874501..1dc1588255 100755
--- git-p4.py
+++ git-p4.py
@@ -1108,8 +1108,8 @@ def getClientSpec():
     # the //client/ name
     client_name = entry["Client"]
 
-    # just the keys that start with "View"
-    view_keys = [ k for k in entry.keys() if k.startswith("View") ]
+    # just the keys "View0", "View1", ...
+    view_keys = [ k for k in entry.keys() if re.match(r"View\d+", k) ]
 
     # hold this new View
     view = View(client_name)

[-- Attachment #4: p4-client-from-p4java.txt --]
[-- Type: text/plain, Size: 3386 bytes --]

  $ p4 -G client -o git-p4-sync-2 | hexdump -C
00000000  7b 73 04 00 00 00 63 6f  64 65 73 04 00 00 00 73  |{s....codes....s|
00000010  74 61 74 73 06 00 00 00  43 6c 69 65 6e 74 73 0d  |tats....Clients.|
00000020  00 00 00 67 69 74 2d 70  34 2d 73 79 6e 63 2d 32  |...git-p4-sync-2|
00000030  73 06 00 00 00 55 70 64  61 74 65 73 13 00 00 00  |s....Updates....|
00000040  32 30 31 39 2f 31 32 2f  31 35 20 31 33 3a 34 39  |2019/12/15 13:49|
00000050  3a 35 31 73 06 00 00 00  41 63 63 65 73 73 73 13  |:51s....Accesss.|
00000060  00 00 00 32 30 31 39 2f  31 32 2f 31 35 20 31 33  |...2019/12/15 13|
00000070  3a 34 39 3a 35 31 73 05  00 00 00 4f 77 6e 65 72  |:49:51s....Owner|
00000080  73 02 00 00 00 73 61 73  04 00 00 00 52 6f 6f 74  |s....sas....Root|
00000090  73 52 00 00 00 2f 76 61  72 2f 61 74 6c 61 73 73  |sR.../var/atlass|
000000a0  69 61 6e 2f 61 70 70 6c  69 63 61 74 69 6f 6e 2d  |ian/application-|
000000b0  64 61 74 61 2f 62 69 74  62 75 63 6b 65 74 2f 73  |data/bitbucket/s|
000000c0  68 61 72 65 64 2f 69 73  2d 67 69 74 2d 70 34 2f  |hared/is-git-p4/|
000000d0  72 65 70 6f 73 69 74 6f  72 79 2f 72 65 70 6f 5f  |repository/repo_|
000000e0  6c 6f 63 61 6c 5f 32 73  07 00 00 00 4f 70 74 69  |local_2s....Opti|
000000f0  6f 6e 73 73 3a 00 00 00  6e 6f 61 6c 6c 77 72 69  |onss:...noallwri|
00000100  74 65 20 6e 6f 63 6c 6f  62 62 65 72 20 6e 6f 63  |te noclobber noc|
00000110  6f 6d 70 72 65 73 73 20  75 6e 6c 6f 63 6b 65 64  |ompress unlocked|
00000120  20 6e 6f 6d 6f 64 74 69  6d 65 20 6e 6f 72 6d 64  | nomodtime normd|
00000130  69 72 73 0d 00 00 00 53  75 62 6d 69 74 4f 70 74  |irs....SubmitOpt|
00000140  69 6f 6e 73 73 0f 00 00  00 73 75 62 6d 69 74 75  |ionss....submitu|
00000150  6e 63 68 61 6e 67 65 64  73 07 00 00 00 4c 69 6e  |nchangeds....Lin|
00000160  65 45 6e 64 73 05 00 00  00 6c 6f 63 61 6c 73 05  |eEnds....locals.|
00000170  00 00 00 56 69 65 77 30  73 1e 00 00 00 2f 2f 72  |...View0s....//r|
00000180  65 70 6f 2f 2e 2e 2e 20  2f 2f 67 69 74 2d 70 34  |epo/... //git-p4|
00000190  2d 73 79 6e 63 2d 32 2f  2e 2e 2e 73 04 00 00 00  |-sync-2/...s....|
000001a0  54 79 70 65 73 09 00 00  00 77 72 69 74 65 61 62  |Types....writeab|
000001b0  6c 65 73 06 00 00 00 42  61 63 6b 75 70 73 06 00  |les....Backups..|
000001c0  00 00 65 6e 61 62 6c 65  73 09 00 00 00 65 78 74  |..enables....ext|
000001d0  72 61 54 61 67 30 73 0d  00 00 00 56 69 65 77 44  |raTag0s....ViewD|
000001e0  65 70 6f 74 54 79 70 65  73 0d 00 00 00 65 78 74  |epotTypes....ext|
000001f0  72 61 54 61 67 54 79 70  65 30 73 04 00 00 00 77  |raTagType0s....w|
00000200  6f 72 64 73 0d 00 00 00  56 69 65 77 44 65 70 6f  |ords....ViewDepo|
00000210  74 54 79 70 65 73 05 00  00 00 67 72 61 70 68 30  |tTypes....graph0|
00000220

  $ p4 -G client -o git-p4-sync-2 | python -c 'import marshal, pprint, sys; pprint.pprint(marshal.load(sys.stdin))'
{'Access': '2019/12/15 13:49:51',
 'Backup': 'enable',
 'Client': 'git-p4-sync-2',
 'LineEnd': 'local',
 'Options': 'noallwrite noclobber nocompress unlocked nomodtime normdir',
 'Owner': 'sa',
 'Root': '/var/atlassian/application-data/bitbucket/shared/is-git-p4/repository/repo_local_2',
 'SubmitOptions': 'submitunchanged',
 'Type': 'writeable',
 'Update': '2019/12/15 13:49:51',
 'View0': '//repo/... //git-p4-sync-2/...',
 'ViewDepotType': 'graph',
 'code': 'stat',
 'extraTag0': 'ViewDepotType',
 'extraTagType0': 'word'}

Re: git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

[Luke Diamand]

On Tue, 7 Jan 2020 at 15:53, Ivan Selin <ivan.selin@toptal.com> wrote:
>
> Hello!
>
> If I create a perforce client from java using p4java, it gets created
> with an extra key "ViewDepotType" in client definition. When I try to
> do `git p4 sync --use-client-spec`, git-p4 dies with message like
> "Expected view key View1 missing" — because it assumes that all keys
> starting with "View" are "View0", "View1" and so on.
>
> How to reproduce:
> 1) Create perforce repository;
> 2) Create a perforce client (let's name it "my-client") in said
> perforce repository using p4java; add one view mapping to the client
> on creation;
> 3) Run `P4CLIENT=my-client git p4 sync --use-client-spec`;
> 4) git p4 finishes with error "Expected view key View1 missing".
>
> Attaching hexdumped/unmarshalled examples of "regular" client
> definition and client created with p4java. Note that p4java's version
> has "ViewDepotType" key and regular client does not. Also,
> "ViewDepotType" key is not showing up in text output of `p4 client
> -o`, only in binary format (`p4 -G client -o`). And I'm attaching a
> patch that solved the issue for me.
>
> Is that information enough or should I add anything else? I'm new to
> git community, but willing to help.

I suspect the problem lies at around line 4220 of git-p4, where it does this:

     view_keys = [ k for k in entry.keys() if k.startswith("View") ]

I think changing that startswith to a regex match would fix this,
although I have not tried it.
Something like:

is_view = re.compile(r'^View\d+$')
view_keys = [ k for k in entry.keys() if is_view.match(k) ]


>
> Regards,
> Ivan.

Re: git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

[Ivan Selin]

Yes, that's exactly the way that I've tried it and it worked for me.
Can we have that integrated into git-p4? What can I do to make it
happen?

On Wed, Jan 29, 2020 at 1:35 PM Luke Diamand <luke@diamand.org> wrote:
>
> On Tue, 7 Jan 2020 at 15:53, Ivan Selin <ivan.selin@toptal.com> wrote:
> >
> > Hello!
> >
> > If I create a perforce client from java using p4java, it gets created
> > with an extra key "ViewDepotType" in client definition. When I try to
> > do `git p4 sync --use-client-spec`, git-p4 dies with message like
> > "Expected view key View1 missing" — because it assumes that all keys
> > starting with "View" are "View0", "View1" and so on.
> >
> > How to reproduce:
> > 1) Create perforce repository;
> > 2) Create a perforce client (let's name it "my-client") in said
> > perforce repository using p4java; add one view mapping to the client
> > on creation;
> > 3) Run `P4CLIENT=my-client git p4 sync --use-client-spec`;
> > 4) git p4 finishes with error "Expected view key View1 missing".
> >
> > Attaching hexdumped/unmarshalled examples of "regular" client
> > definition and client created with p4java. Note that p4java's version
> > has "ViewDepotType" key and regular client does not. Also,
> > "ViewDepotType" key is not showing up in text output of `p4 client
> > -o`, only in binary format (`p4 -G client -o`). And I'm attaching a
> > patch that solved the issue for me.
> >
> > Is that information enough or should I add anything else? I'm new to
> > git community, but willing to help.
>
> I suspect the problem lies at around line 4220 of git-p4, where it does this:
>
>      view_keys = [ k for k in entry.keys() if k.startswith("View") ]
>
> I think changing that startswith to a regex match would fix this,
> although I have not tried it.
> Something like:
>
> is_view = re.compile(r'^View\d+$')
> view_keys = [ k for k in entry.keys() if is_view.match(k) ]
>
>
> >
> > Regards,
> > Ivan.

Re: git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

[Luke Diamand]

On Fri, 31 Jan 2020 at 13:43, Ivan Selin <ivan.selin@toptal.com> wrote:
>
> Yes, that's exactly the way that I've tried it and it worked for me.
> Can we have that integrated into git-p4? What can I do to make it
> happen?

A patch would be very welcome!

See Documentation/SubmittingPatches for details, but it's likely
enough to just make the change, and then use git format-patch and git
send-email.

Please ensure you add a "Signed-off-by" (git commit -s).

Thanks!
Luke


>
> On Wed, Jan 29, 2020 at 1:35 PM Luke Diamand <luke@diamand.org> wrote:
> >
> > On Tue, 7 Jan 2020 at 15:53, Ivan Selin <ivan.selin@toptal.com> wrote:
> > >
> > > Hello!
> > >
> > > If I create a perforce client from java using p4java, it gets created
> > > with an extra key "ViewDepotType" in client definition. When I try to
> > > do `git p4 sync --use-client-spec`, git-p4 dies with message like
> > > "Expected view key View1 missing" — because it assumes that all keys
> > > starting with "View" are "View0", "View1" and so on.
> > >
> > > How to reproduce:
> > > 1) Create perforce repository;
> > > 2) Create a perforce client (let's name it "my-client") in said
> > > perforce repository using p4java; add one view mapping to the client
> > > on creation;
> > > 3) Run `P4CLIENT=my-client git p4 sync --use-client-spec`;
> > > 4) git p4 finishes with error "Expected view key View1 missing".
> > >
> > > Attaching hexdumped/unmarshalled examples of "regular" client
> > > definition and client created with p4java. Note that p4java's version
> > > has "ViewDepotType" key and regular client does not. Also,
> > > "ViewDepotType" key is not showing up in text output of `p4 client
> > > -o`, only in binary format (`p4 -G client -o`). And I'm attaching a
> > > patch that solved the issue for me.
> > >
> > > Is that information enough or should I add anything else? I'm new to
> > > git community, but willing to help.
> >
> > I suspect the problem lies at around line 4220 of git-p4, where it does this:
> >
> >      view_keys = [ k for k in entry.keys() if k.startswith("View") ]
> >
> > I think changing that startswith to a regex match would fix this,
> > although I have not tried it.
> > Something like:
> >
> > is_view = re.compile(r'^View\d+$')
> > view_keys = [ k for k in entry.keys() if is_view.match(k) ]
> >
> >
> > >
> > > Regards,
> > > Ivan.

Re: git-p4 cannot use perforce client created by p4java — "Expected view key View1 missing"

[Junio C Hamano]

Luke Diamand <luke@diamand.org> writes:

> On Fri, 31 Jan 2020 at 13:43, Ivan Selin <ivan.selin@toptal.com> wrote:
>>
>> Yes, that's exactly the way that I've tried it and it worked for me.
>> Can we have that integrated into git-p4? What can I do to make it
>> happen?
>
> A patch would be very welcome!
>
> See Documentation/SubmittingPatches for details, but it's likely
> enough to just make the change, and then use git format-patch and git
> send-email.
>
> Please ensure you add a "Signed-off-by" (git commit -s).

Thanks for helping.  Another area a new person struggles is to
propose a good commit log message.  I'd expect this change to be
explained something like this:

    Subject: git-p4: tighten detection of view names

    The current "git p4" code expects that all keys starting with
    "View" are view names (i.e. "View0", "View1", etc.), but a
    Perforce client created using p4java gets an extra key
    "ViewDepotType", which is not something we want to be treated as
    a view key.

    Tighten the filter of keys to accept only string "View" followed
    by nothing but digits.

    Helped-by: Luke Diamand <luke@diamand.org>
    Signed-off-by: ...

Note that I am not a Perforce person, so the phrase "view names" I
used in the above may totally be wrong as P4-lingo, in which case
please replace them with what is acceptable by our Perforce friends
;-)

Thanks.

>> ...
>> > is_view = re.compile(r'^View\d+$')
>> > view_keys = [ k for k in entry.keys() if is_view.match(k) ]