Re: git p4 clone not processing branches properly

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

Hi,

I'm trying to convert a Perforce repository to git, knowing that:
- I use client specs to remove a bunch of folders containing binaires
(several GiB)
- branch mappings may not be properly set, and I can't change them

Now, the branches are layout like this:
- Branch/Main <- master
- Branch/Feature1
- ...
- Releases/2013
- ...
I would like to have these branches and releases branches inside by
cloned git repository, but this doesn't work. I keep on getting each
file with Project/Branch/Main as well as Project/Branch/Feature1 and
all others in my master branch.
I tried to add branchLists like this:
        branchList = Branch/Main:Releases/2013
        branchList = Releases/2013:Branch/Feature1
but it doesn't change a thing with the following command:
 git p4 clone --verbose --use-client-spec --detect-branches
//Depot/Project@specificrevision

I can see that branches are detected from the Perforce server, but
none are actually detected for this specific project:
p4-git branches: []
initial parents: {}

Can someone give a pointer to a tutorial or something for a complex
case like this?

Regards,

Matthieu Brucher
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Vitor Antunes]

Matthieu Brucher <matthieu.brucher <at> gmail.com> writes:
> 
> Hi,
> 
> I'm trying to convert a Perforce repository to git, knowing that:
> - I use client specs to remove a bunch of folders containing binaires
> (several GiB)
> - branch mappings may not be properly set, and I can't change them
> 
> Now, the branches are layout like this:
> - Branch/Main <- master
> - Branch/Feature1
> - ...
> - Releases/2013
> - ...
> I would like to have these branches and releases branches inside by
> cloned git repository, but this doesn't work. I keep on getting each
> file with Project/Branch/Main as well as Project/Branch/Feature1 and
> all others in my master branch.
> I tried to add branchLists like this:
>         branchList = Branch/Main:Releases/2013
>         branchList = Releases/2013:Branch/Feature1
> but it doesn't change a thing with the following command:
>  git p4 clone --verbose --use-client-spec --detect-branches
> //Depot/Project <at> specificrevision
> 
> I can see that branches are detected from the Perforce server, but
> none are actually detected for this specific project:
> p4-git branches: []
> initial parents: {}
> 
> Can someone give a pointer to a tutorial or something for a complex
> case like this?

Hi Matthieu,

Could you please try using //Depot/Project<at>all instead of selecting a
specific revision? 
Also, by using that command it means that the following depot paths must
exist:
//Depot/Project/Branch/Main
//Depot/Project/Releases/2013
//Depot/Project/Branch/Feature1

I've never used the --use-client-spec, so I'm not sure if that will not
break the branch detection code.

Cheers,
Vitor

P.S. - Please keep me in CC because I'm not subscribed to the mailing
list.

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

> Hi Matthieu,
>
> Could you please try using //Depot/Project<at>all instead of selecting a
> specific revision?

I can try. Indeed, at this revision, the two other branches do not yet
exist. But @all will get everything? Last time, I only got head
(IIRC).

> Also, by using that command it means that the following depot paths must
> exist:
> //Depot/Project/Branch/Main
> //Depot/Project/Releases/2013
> //Depot/Project/Branch/Feature1

Yes, they indeed do.

> I've never used the --use-client-spec, so I'm not sure if that will not
> break the branch detection code.

I need to do that because if I don't, the depot is clobbed with
binaries. Or perhaps if I put some .gitignore stuff, I might not do
this?

> Cheers,
> Vitor

Thanks for the tips, I will try tomorrow.

Cheers,
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Vitor Antunes]

On Fri, Jul 5, 2013 at 7:11 PM, Matthieu Brucher
<matthieu.brucher@gmail.com> wrote:
>> Hi Matthieu,
>>
>> Could you please try using //Depot/Project<at>all instead of selecting a
>> specific revision?
>
> I can try. Indeed, at this revision, the two other branches do not yet
> exist. But @all will get everything? Last time, I only got head
> (IIRC).

Our P4 server has a limitation on the number of lines returned by "p4
changes" command, so sometimes I have to use @change_start,@change_stop
instead of @all. You might want to use this range limitation to test
git-p4 by limiting to a small number of changelists that allows you to
check if at least one branch is correctly detected.

>> Also, by using that command it means that the following depot paths must
>> exist:
>> //Depot/Project/Branch/Main
>> //Depot/Project/Releases/2013
>> //Depot/Project/Branch/Feature1
>
> Yes, they indeed do.

In this case the problem should not be in branchList configuration.

>> I've never used the --use-client-spec, so I'm not sure if that will not
>> break the branch detection code.
>
> I need to do that because if I don't, the depot is clobbed with
> binaries. Or perhaps if I put some .gitignore stuff, I might not do
> this?

Keep using it, at least for now. If everything else fails we can look at
this again.

Cheers,
Vitor

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

>> I can try. Indeed, at this revision, the two other branches do not yet
>> exist. But @all will get everything? Last time, I only got head
>> (IIRC).
>
> Our P4 server has a limitation on the number of lines returned by "p4
> changes" command, so sometimes I have to use @change_start,@change_stop
> instead of @all. You might want to use this range limitation to test
> git-p4 by limiting to a small number of changelists that allows you to
> check if at least one branch is correctly detected.

I didn't know about this. I wanted to start the cloning at some point
in the past, that's why I used the @123456789 notation.

>>> Also, by using that command it means that the following depot paths must
>>> exist:
>>> //Depot/Project/Branch/Main
>>> //Depot/Project/Releases/2013
>>> //Depot/Project/Branch/Feature1
>>
>> Yes, they indeed do.
>
> In this case the problem should not be in branchList configuration.
>
>>> I've never used the --use-client-spec, so I'm not sure if that will not
>>> break the branch detection code.
>>
>> I need to do that because if I don't, the depot is clobbed with
>> binaries. Or perhaps if I put some .gitignore stuff, I might not do
>> this?
>
> Keep using it, at least for now. If everything else fails we can look at
> this again.

OK, I'll send a mail on Monday (forgot it was the week end tomorrow...)

Cheers,

Matthieu
--
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

Hi again,

I tried with @all, but it didn'y work as expected. It imported a bunch
of revisions (but no files?) and ended with:
  Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
  Not checking out any branch, use "git checkout -q -b master <branch>"
  executing git config --bool git-p4.useclientspec true

And when I tried to checkout Branch/Main, it failed with
  fatal: Cannot update paths and switch to branch 'master' at the same time.
  Did you intend to checkout 'Branch/Main' which can not be resolved as commit?

Thanks,

Matthieu


2013/7/5 Matthieu Brucher <matthieu.brucher@gmail.com>:
>>> I can try. Indeed, at this revision, the two other branches do not yet
>>> exist. But @all will get everything? Last time, I only got head
>>> (IIRC).
>>
>> Our P4 server has a limitation on the number of lines returned by "p4
>> changes" command, so sometimes I have to use @change_start,@change_stop
>> instead of @all. You might want to use this range limitation to test
>> git-p4 by limiting to a small number of changelists that allows you to
>> check if at least one branch is correctly detected.
>
> I didn't know about this. I wanted to start the cloning at some point
> in the past, that's why I used the @123456789 notation.
>
>>>> Also, by using that command it means that the following depot paths must
>>>> exist:
>>>> //Depot/Project/Branch/Main
>>>> //Depot/Project/Releases/2013
>>>> //Depot/Project/Branch/Feature1
>>>
>>> Yes, they indeed do.
>>
>> In this case the problem should not be in branchList configuration.
>>
>>>> I've never used the --use-client-spec, so I'm not sure if that will not
>>>> break the branch detection code.
>>>
>>> I need to do that because if I don't, the depot is clobbed with
>>> binaries. Or perhaps if I put some .gitignore stuff, I might not do
>>> this?
>>
>> Keep using it, at least for now. If everything else fails we can look at
>> this again.
>
> OK, I'll send a mail on Monday (forgot it was the week end tomorrow...)
>
> Cheers,
>
> Matthieu
> --
> Information System Engineer, Ph.D.
> Blog: http://matt.eifelle.com
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
> Music band: http://liliejay.com/



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Vitor Antunes]

On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher
<matthieu.brucher@gmail.com> wrote:
> Hi again,
>
> I tried with @all, but it didn'y work as expected. It imported a bunch
> of revisions (but no files?) and ended with:
>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>   Not checking out any branch, use "git checkout -q -b master <branch>"
>   executing git config --bool git-p4.useclientspec true
>
> And when I tried to checkout Branch/Main, it failed with
>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?

Hi Matthieu,

Please run "git branch -a" in that repository and you should be able
to see the various branches under /remotes/p4/
Then you just need to choose a branch and run "git checkout -b
git_branch_name p4/p4_branch_name".

Cheers,
Vitor

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

Unfortunately, git branch -a returns nothing :/
I tried with the simple detect-branches as well as with the config values.
Perhaps the spec-client? Although it is strange as it seems that the
repository is completely empty.

Thanks,

Matthieu

2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
> On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher
> <matthieu.brucher@gmail.com> wrote:
>> Hi again,
>>
>> I tried with @all, but it didn'y work as expected. It imported a bunch
>> of revisions (but no files?) and ended with:
>>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>>   Not checking out any branch, use "git checkout -q -b master <branch>"
>>   executing git config --bool git-p4.useclientspec true
>>
>> And when I tried to checkout Branch/Main, it failed with
>>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
>
> Hi Matthieu,
>
> Please run "git branch -a" in that repository and you should be able
> to see the various branches under /remotes/p4/
> Then you just need to choose a branch and run "git checkout -b
> git_branch_name p4/p4_branch_name".
>
> Cheers,
> Vitor



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

Without the spec client, it seems that the branches are recognized,
but there are some many binary files that I need to remove them during
the migration.
I tried setting a .gitignore beforehand, but it is not respected (I
tried to remove some folders with folder/ in .gitignore, but the
folder are still imported).
It there a switch for the import somewhere?

Thanks,

Matthieu

2013/7/8 Matthieu Brucher <matthieu.brucher@gmail.com>:
> Unfortunately, git branch -a returns nothing :/
> I tried with the simple detect-branches as well as with the config values.
> Perhaps the spec-client? Although it is strange as it seems that the
> repository is completely empty.
>
> Thanks,
>
> Matthieu
>
> 2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
>> On Mon, Jul 8, 2013 at 11:09 AM, Matthieu Brucher
>> <matthieu.brucher@gmail.com> wrote:
>>> Hi again,
>>>
>>> I tried with @all, but it didn'y work as expected. It imported a bunch
>>> of revisions (but no files?) and ended with:
>>>   Reading pipe: ['git', 'config', '--bool', 'git-p4.importLabels']
>>>   Not checking out any branch, use "git checkout -q -b master <branch>"
>>>   executing git config --bool git-p4.useclientspec true
>>>
>>> And when I tried to checkout Branch/Main, it failed with
>>>   fatal: Cannot update paths and switch to branch 'master' at the same time.
>>>   Did you intend to checkout 'Branch/Main' which can not be resolved as commit?
>>
>> Hi Matthieu,
>>
>> Please run "git branch -a" in that repository and you should be able
>> to see the various branches under /remotes/p4/
>> Then you just need to choose a branch and run "git checkout -b
>> git_branch_name p4/p4_branch_name".
>>
>> Cheers,
>> Vitor
>
>
>
> --
> Information System Engineer, Ph.D.
> Blog: http://matt.eifelle.com
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
> Music band: http://liliejay.com/



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Vitor Antunes]

On Mon, Jul 8, 2013 at 12:10 PM, Matthieu Brucher
<matthieu.brucher@gmail.com> wrote:
> Without the spec client, it seems that the branches are recognized,
> but there are some many binary files that I need to remove them during
> the migration.
> I tried setting a .gitignore beforehand, but it is not respected (I
> tried to remove some folders with folder/ in .gitignore, but the
> folder are still imported).
> It there a switch for the import somewhere?

Hi Matthieu,

Unfortunately I've never tested the branch detection together with spec
configuration. But there is a test case for it in the code that refers
to the following question in StackOverflow:

http://stackoverflow.com/questions/11893688

Could you also tell us which version of git you are using?

Pete, maybe you can help Matthieu further on this question?

Thanks,
Vitor

Re: git p4 clone not processing branches properly

[Matthieu Brucher]

Hi,

I tried without spec, but then it tried importing everything, even
though there was a .gitignore and a .git/config/exclude file.
Then, it crashed during the importation because it could find an old
branch (I don't have access to everything on the repository), so I
tried importing just the recent past, but then it failed because it
identified a branch names Branch/Main/src...
It is starting to feel as if I will have to compromise between
something that works but without branches and without the proper names
(the files are named Project/Branch/Main/...) or having the proper
names, but with all binaries, bogus branches...
I know it is not due to git, it is mainly that Perforce and git have
very different workflows. Or perhaps with any luck, the server is up
to date, and I can find a way of using Perforce's bridge.

Thanks,

Matthieu



2013/7/8 Vitor Antunes <vitor.hda@gmail.com>:
> On Mon, Jul 8, 2013 at 12:10 PM, Matthieu Brucher
> <matthieu.brucher@gmail.com> wrote:
>> Without the spec client, it seems that the branches are recognized,
>> but there are some many binary files that I need to remove them during
>> the migration.
>> I tried setting a .gitignore beforehand, but it is not respected (I
>> tried to remove some folders with folder/ in .gitignore, but the
>> folder are still imported).
>> It there a switch for the import somewhere?
>
> Hi Matthieu,
>
> Unfortunately I've never tested the branch detection together with spec
> configuration. But there is a test case for it in the code that refers
> to the following question in StackOverflow:
>
> http://stackoverflow.com/questions/11893688
>
> Could you also tell us which version of git you are using?
>
> Pete, maybe you can help Matthieu further on this question?
>
> Thanks,
> Vitor



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

Re: git p4 clone not processing branches properly

[Vitor Antunes]

On Mon, Jul 8, 2013 at 3:03 PM, Matthieu Brucher
<matthieu.brucher@gmail.com> wrote:
> Hi,
>
> I tried without spec, but then it tried importing everything, even
> though there was a .gitignore and a .git/config/exclude file.
> Then, it crashed during the importation because it could find an old
> branch (I don't have access to everything on the repository), so I
> tried importing just the recent past, but then it failed because it
> identified a branch names Branch/Main/src...
> It is starting to feel as if I will have to compromise between
> something that works but without branches and without the proper names
> (the files are named Project/Branch/Main/...) or having the proper
> names, but with all binaries, bogus branches...
> I know it is not due to git, it is mainly that Perforce and git have
> very different workflows. Or perhaps with any luck, the server is up
> to date, and I can find a way of using Perforce's bridge.

Hi Matthieu,

I created a simple test case where I added a file to one of the branches in
P4 but excluded it in the client spec. During synchronization git p4
correctly ignored it.

Please check which git version you are using. It is possible the issue you
are experiencing was fixed in a more recent version.

Also, if possible, please try creating a simple test case that replicates
your issue (p4 and p4d are free to use when the repo is only used by one
person). If you record the commands you used and share them with me I will
be able to better check this issue on my side.

Cheers,
Vitor