git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* State of Perforce importing.
@ 2007-09-17 19:30 David Brown
  2007-09-18  6:58 ` Simon Hausmann
                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: David Brown @ 2007-09-17 19:30 UTC (permalink / raw)
  To: Git

I'd like to track a lot of code living in a Perforce repository, so I've
been playing with 'git-p4.py'.  Is the one in the contrib/fast-import
directory the latest version, or is there a better place.

So far, it is having a couple of problems:

   - The commit comment is empty.  It doesn't seem to grab the Perforce
     description, and the user seems to be <a@b>.

   - Every revision seems to check every file out of Perforce.  This means
     that for the directory I want, every revision is going to take about 20
     minutes.

Before I start working on it, I'd like to make sure I'm working on the
latest code, though.

Thanks,
David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-17 19:30 State of Perforce importing David Brown
@ 2007-09-18  6:58 ` Simon Hausmann
  2007-09-18  7:27 ` Sam Vilain
  2007-09-18 23:37 ` David Brown
  2 siblings, 0 replies; 19+ messages in thread
From: Simon Hausmann @ 2007-09-18  6:58 UTC (permalink / raw)
  To: David Brown; +Cc: Git

[-- Attachment #1: Type: text/plain, Size: 1094 bytes --]

On Monday 17 September 2007 21:30:28 David Brown wrote:
> I'd like to track a lot of code living in a Perforce repository, so I've
> been playing with 'git-p4.py'.  Is the one in the contrib/fast-import
> directory the latest version, or is there a better place.

This is indeed the latest version (on contrib/fast-import).

> So far, it is having a couple of problems:
>
>    - The commit comment is empty.  It doesn't seem to grab the Perforce
>      description, and the user seems to be <a@b>.

This may be a problem with the python output of perforce. Can you run the 
following command?

	git-p4 debug change <a change number in your depot>

That should print a dictionary that has a 'desc' field containing the commit 
comment/log and a 'user' field that has the perforce user name.

>    - Every revision seems to check every file out of Perforce.  This means
>      that for the directory I want, every revision is going to take about
> 20 minutes.

For every revision only every _changed_ file is retrieved (using p4 
print //path/file#revision).


Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-17 19:30 State of Perforce importing David Brown
  2007-09-18  6:58 ` Simon Hausmann
@ 2007-09-18  7:27 ` Sam Vilain
  2007-09-18 15:49   ` David Brown
  2007-09-18 23:37 ` David Brown
  2 siblings, 1 reply; 19+ messages in thread
From: Sam Vilain @ 2007-09-18  7:27 UTC (permalink / raw)
  To: David Brown; +Cc: Git

David Brown wrote:
> I'd like to track a lot of code living in a Perforce repository, so I've
> been playing with 'git-p4.py'.  Is the one in the contrib/fast-import
> directory the latest version, or is there a better place.

I'm pretty close to giving a newer one a spin, that actually imports
from the raw perforce back-end files without needing the perforce
server.  I am hoping that this should give a very clean import and will
be very fast and efficient, sending files that share ancestry to gfi in
sequence so that the on-the-fly delta system works.

If you're interested, take a look at
http://utsl.gen.nz/gitweb/?p=git-p4raw;a=summary.  Expect the commands
that say "WIP" to be rebased :-).  It requires Postgres - I haven't yet
re-written the SQL queries that step outside of MySQL's little box.

It could possibly be adapted to use the p4 client (though I'd expect
that to be relatively slow per-revision), and possibly be extended to be
bidirectional as all of the upstream change number information is
recorded, a la git-svn.

Sam.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18  7:27 ` Sam Vilain
@ 2007-09-18 15:49   ` David Brown
  2007-09-18 17:53     ` Reece Dunn
  2007-09-20  6:12     ` Dmitry Kakurin
  0 siblings, 2 replies; 19+ messages in thread
From: David Brown @ 2007-09-18 15:49 UTC (permalink / raw)
  To: Sam Vilain; +Cc: Git

On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:

>I'm pretty close to giving a newer one a spin, that actually imports
>from the raw perforce back-end files without needing the perforce
>server.  I am hoping that this should give a very clean import and will
>be very fast and efficient, sending files that share ancestry to gfi in
>sequence so that the on-the-fly delta system works.

Unfortunately, this isn't something I'm going to be able to use.  The
Perforce server will remain live, and resides on a machine I don't have
access to.

>It could possibly be adapted to use the p4 client (though I'd expect
>that to be relatively slow per-revision), and possibly be extended to be
>bidirectional as all of the upstream change number information is
>recorded, a la git-svn.

I was able to get 'git-p4' to work a lot better by using @all, but it still
has some problems, at least bad interactions with P4.

   - It doesn't use any client spec.  Our P4 server space is a complete
     mismash and has to be fixed up to get a sane directory layout.  For
     example, some revisions have hundred-MB tar files sitting in the root
     directory and I don't want that in the repo.  I also need to exclude
     directories, and in some cases completely rearrange the directory
     layout.

   - Our P4 server is set to be case insensitive.  'git-p4' ignores paths
     that come back from the server that are specified using a different
     case.  Unfortunately, this means that a handful of files just get
     randomly dropped from each revision.

     I tried importing a client path instead of a depot path, but the names
     that come back from 'p4 files' are depot based so none ever match.  I
     end up with a nice revision history of entirely empty trees.

I'm probably going to end up writing an importer that uses an actual client
workspace to let Perforce do the client mapping.  I'm also going to have to
put some work into some code to clean up the log messages, since most of
our changes have as a first line "New Features:", which makes for a rather
uninformative shortlog.

But, I did learn about 'p4 -G' from git-p4 so that will help in getting
information from the repository.

Thanks,
David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 15:49   ` David Brown
@ 2007-09-18 17:53     ` Reece Dunn
  2007-09-18 23:19       ` David Brown
  2007-09-19  0:23       ` Sam Vilain
  2007-09-20  6:12     ` Dmitry Kakurin
  1 sibling, 2 replies; 19+ messages in thread
From: Reece Dunn @ 2007-09-18 17:53 UTC (permalink / raw)
  To: Sam Vilain, Git

On 18/09/2007, David Brown <git@davidb.org> wrote:
> On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:
>
> >I'm pretty close to giving a newer one a spin, that actually imports
> >from the raw perforce back-end files without needing the perforce
> >server.  I am hoping that this should give a very clean import and will
> >be very fast and efficient, sending files that share ancestry to gfi in
> >sequence so that the on-the-fly delta system works.
>
> Unfortunately, this isn't something I'm going to be able to use.  The
> Perforce server will remain live, and resides on a machine I don't have
> access to.

I use git-p4 in the same way. The best approach would be to have both
tools and to use whichever one best matches your needs.

> >It could possibly be adapted to use the p4 client (though I'd expect
> >that to be relatively slow per-revision), and possibly be extended to be
> >bidirectional as all of the upstream change number information is
> >recorded, a la git-svn.
>
> I was able to get 'git-p4' to work a lot better by using @all, but it still
> has some problems, at least bad interactions with P4.

I have also seen this.

>    - It doesn't use any client spec.  Our P4 server space is a complete
>      mismash and has to be fixed up to get a sane directory layout.  For
>      example, some revisions have hundred-MB tar files sitting in the root
>      directory and I don't want that in the repo.  I also need to exclude
>      directories, and in some cases completely rearrange the directory
>      layout.

The directory exclusion you could do the other way, if git-p4
supported multiple directory paths.

The main issues with using client workspaces is that they require you
to use `p4 sync`, whereas git-p4 uses `p4 print` and that they may
change as the repository changes, but Perforce does not track these
changes.

That said, something like what workspaces are doing (allowing you to
specify multiple paths and where they are to go) would be useful.

>    - Our P4 server is set to be case insensitive.  'git-p4' ignores paths
>      that come back from the server that are specified using a different
>      case.  Unfortunately, this means that a handful of files just get
>      randomly dropped from each revision.

It is worse than this. If you have:

    p4 integrate foo Foo
    p4 delete foo

then git-p4 will completely remove the file foo from the repository! I
reported this a while back, but did not get a reply.

This is something I want to fix, as doing an '@all', rebase or sync is
broken in this case when dealing with renamed files as above when
importing a repository on a case insensitive system.

The alternative is to do the importing from Perforce on a Linux
machine and then clone/pull/rebase from it on a Windows machine.

>      I tried importing a client path instead of a depot path, but the names
>      that come back from 'p4 files' are depot based so none ever match.  I
>      end up with a nice revision history of entirely empty trees.

Ideally, git-p4 should bail out here with an error about the path
needing to be specified as a depot path.

> I'm probably going to end up writing an importer that uses an actual client
> workspace to let Perforce do the client mapping.

As eluded to above, I want to extend git-p4 to support a
workspace-like importer map file to use instead of a specific depot
path (with a single depot path being supported as well for backward
compatibility and simplicity if the repository you are importing from
has a simple layout).

There is no need to create yet another Perforce importing tool, git-p4
works well in most cases. If we focus on improving git-p4, extending
it to support the functionality mentioned here, fix the issues that
there are with it, then that will be more beneficial to the community
as they will not have to learn another tool with a different set of
bugs and issues.

> I'm also going to have to
> put some work into some code to clean up the log messages, since most of
> our changes have as a first line "New Features:", which makes for a rather
> uninformative shortlog.

I would not do that. It is a good idea to keep the original log
messages, even if it does make for an uninformative shortlog. Look at
some of the CVS/SVN imported logs!

- Reece

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 17:53     ` Reece Dunn
@ 2007-09-18 23:19       ` David Brown
  2007-09-19  0:20         ` Sam Vilain
  2007-09-19  0:23       ` Sam Vilain
  1 sibling, 1 reply; 19+ messages in thread
From: David Brown @ 2007-09-18 23:19 UTC (permalink / raw)
  To: Reece Dunn; +Cc: Sam Vilain, Git

On Tue, Sep 18, 2007 at 06:53:45PM +0100, Reece Dunn wrote:

>The main issues with using client workspaces is that they require you
>to use `p4 sync`, whereas git-p4 uses `p4 print` and that they may
>change as the repository changes, but Perforce does not track these
>changes.

Unfortunately, we have one project that heavily abuses P4 client specs.
For every release, someone creates a >900 line client spec and labels the
files in it.  Those are the versions that need to get checked in, and
without rewriting much of what P4 does, I'm going to have to let P4 do the
syncing and checking out.

>I would not do that. It is a good idea to keep the original log
>messages, even if it does make for an uninformative shortlog. Look at
>some of the CVS/SVN imported logs!

I think what I want then is something to filter between 'git log' and 'git
shortlog' that would find a summary line in the commit message and copy it
to the top.  It wouldn't change the history, but clean it up for shortlog's
purpose.

David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-17 19:30 State of Perforce importing David Brown
  2007-09-18  6:58 ` Simon Hausmann
  2007-09-18  7:27 ` Sam Vilain
@ 2007-09-18 23:37 ` David Brown
  2007-09-19  0:23   ` Sam Vilain
  2007-09-19  6:19   ` Simon Hausmann
  2 siblings, 2 replies; 19+ messages in thread
From: David Brown @ 2007-09-18 23:37 UTC (permalink / raw)
  To: Git

On Mon, Sep 17, 2007 at 12:30:28PM -0700, David Brown wrote:
> I'd like to track a lot of code living in a Perforce repository, so I've
> been playing with 'git-p4.py'.  Is the one in the contrib/fast-import
> directory the latest version, or is there a better place.
>
> So far, it is having a couple of problems:
>
>   - The commit comment is empty.  It doesn't seem to grab the Perforce
>     description, and the user seems to be <a@b>.
>
>   - Every revision seems to check every file out of Perforce.  This means
>     that for the directory I want, every revision is going to take about 20
>     minutes.

An additional problem:

   - git-p4 doesn't preserve the execute permission bit from Perforce.

David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 23:19       ` David Brown
@ 2007-09-19  0:20         ` Sam Vilain
  2007-09-19  0:26           ` David Brown
  0 siblings, 1 reply; 19+ messages in thread
From: Sam Vilain @ 2007-09-19  0:20 UTC (permalink / raw)
  To: Reece Dunn; +Cc: Git

David Brown wrote:
> On Tue, Sep 18, 2007 at 06:53:45PM +0100, Reece Dunn wrote:
> 
>> The main issues with using client workspaces is that they require you
>> to use `p4 sync`, whereas git-p4 uses `p4 print` and that they may
>> change as the repository changes, but Perforce does not track these
>> changes.
> 
> Unfortunately, we have one project that heavily abuses P4 client specs.
> For every release, someone creates a >900 line client spec and labels the
> files in it.  Those are the versions that need to get checked in, and
> without rewriting much of what P4 does, I'm going to have to let P4 do the
> syncing and checking out.

If you can get a hold of the "checkpoint" and "journal" files, you could
probably throw the client spec data into a few Pg tables, chuck a couple
of constraints on it to confirm that it works the way you thought, and
then get the information on what's where using a SQL query.  The file
images themselves can come from wherever, it doesn't really matter
because there are MD5 hashes in the data tables you can use to confirm
you got the right file.

I haven't looked at importing the client spec because it's not important
for the project I'm importing.  But I'd be happy to provide pointers.

>> I would not do that. It is a good idea to keep the original log
>> messages, even if it does make for an uninformative shortlog. Look at
>> some of the CVS/SVN imported logs!
> I think what I want then is something to filter between 'git log' and 'git
> shortlog' that would find a summary line in the commit message and copy it
> to the top.  It wouldn't change the history, but clean it up for shortlog's
> purpose.

Sounds like a job for a templating git-log porcelain.

Sam.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 23:37 ` David Brown
@ 2007-09-19  0:23   ` Sam Vilain
  2007-09-19  0:27     ` David Brown
  2007-09-19  6:19   ` Simon Hausmann
  1 sibling, 1 reply; 19+ messages in thread
From: Sam Vilain @ 2007-09-19  0:23 UTC (permalink / raw)
  To: David Brown; +Cc: Git

David Brown wrote:
> 
> An additional problem:
> 
>    - git-p4 doesn't preserve the execute permission bit from Perforce.

FWIW I found that bit on bit 9 of the 'file type' flag in the db, which
is the third column in the "db.rev" table.  It's used to come up with
the silly names like "text" vs "xtext" (difference?  well, one's
executable of course).

Sam.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 17:53     ` Reece Dunn
  2007-09-18 23:19       ` David Brown
@ 2007-09-19  0:23       ` Sam Vilain
  2007-09-19 21:20         ` Reece Dunn
  1 sibling, 1 reply; 19+ messages in thread
From: Sam Vilain @ 2007-09-19  0:23 UTC (permalink / raw)
  To: Reece Dunn; +Cc: Git

Reece Dunn wrote:
> There is no need to create yet another Perforce importing tool, git-p4
> works well in most cases. If we focus on improving git-p4, extending
> it to support the functionality mentioned here, fix the issues that
> there are with it, then that will be more beneficial to the community
> as they will not have to learn another tool with a different set of
> bugs and issues.

I like my approach; it's clean and I think shows a tasteful level of
distrust towards the sanity and integrity of the data held by Perforce.
 Actually it really helped me understand what was really going on;
because the information as displayed by for instance "p4 integrate" is
a lot more confusing than the underlying tables (IMHO).

Sam.

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19  0:20         ` Sam Vilain
@ 2007-09-19  0:26           ` David Brown
  0 siblings, 0 replies; 19+ messages in thread
From: David Brown @ 2007-09-19  0:26 UTC (permalink / raw)
  To: Sam Vilain; +Cc: Reece Dunn, Git

On Wed, Sep 19, 2007 at 12:20:44PM +1200, Sam Vilain wrote:

>If you can get a hold of the "checkpoint" and "journal" files, you could
>probably throw the client spec data into a few Pg tables, chuck a couple
>of constraints on it to confirm that it works the way you thought, and
>then get the information on what's where using a SQL query.  The file
>images themselves can come from wherever, it doesn't really matter
>because there are MD5 hashes in the data tables you can use to confirm
>you got the right file.

In my instance, I don't have an account on the P4 server, so I'm going to
have to deal with this through P4's normal client.

I don't have much confidence that P4 really knows what it is doing when it
comes to integrate, so I'm not much worried about branches.  But, I do want
to accurately get the history of a particular branch.

So far, the only thing that isn't working is the execute bit doesn't get
set on files that should have it.

Dave

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19  0:23   ` Sam Vilain
@ 2007-09-19  0:27     ` David Brown
  0 siblings, 0 replies; 19+ messages in thread
From: David Brown @ 2007-09-19  0:27 UTC (permalink / raw)
  To: Sam Vilain; +Cc: Git

On Wed, Sep 19, 2007 at 12:23:40PM +1200, Sam Vilain wrote:
>David Brown wrote:
>> 
>> An additional problem:
>> 
>>    - git-p4 doesn't preserve the execute permission bit from Perforce.
>
>FWIW I found that bit on bit 9 of the 'file type' flag in the db, which
>is the third column in the "db.rev" table.  It's used to come up with
>the silly names like "text" vs "xtext" (difference?  well, one's
>executable of course).

It does come back in the 'kind' field when it asks the client for the file
type.  I'll look into using that information to set the execute bit in the
mode it sends off.

Dave

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 23:37 ` David Brown
  2007-09-19  0:23   ` Sam Vilain
@ 2007-09-19  6:19   ` Simon Hausmann
  2007-09-19 17:12     ` David Brown
  1 sibling, 1 reply; 19+ messages in thread
From: Simon Hausmann @ 2007-09-19  6:19 UTC (permalink / raw)
  To: David Brown; +Cc: Git

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]

On Wednesday 19 September 2007 01:37:49 David Brown wrote:
> On Mon, Sep 17, 2007 at 12:30:28PM -0700, David Brown wrote:
> > I'd like to track a lot of code living in a Perforce repository, so I've
> > been playing with 'git-p4.py'.  Is the one in the contrib/fast-import
> > directory the latest version, or is there a better place.
> >
> > So far, it is having a couple of problems:
> >
> >   - The commit comment is empty.  It doesn't seem to grab the Perforce
> >     description, and the user seems to be <a@b>.
> >
> >   - Every revision seems to check every file out of Perforce.  This means
> >     that for the directory I want, every revision is going to take about
> > 20 minutes.
>
> An additional problem:
>
>    - git-p4 doesn't preserve the execute permission bit from Perforce.

Hmm, can you paste the output of

	p4 fstat //path/in/depot/to/file/that/is/imported/incorrectly

? I'm interested in the type of the file that p4 reports.

FWIW it works for me ;-)

Thanks,
Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19  6:19   ` Simon Hausmann
@ 2007-09-19 17:12     ` David Brown
  2007-09-19 18:23       ` Reece Dunn
  0 siblings, 1 reply; 19+ messages in thread
From: David Brown @ 2007-09-19 17:12 UTC (permalink / raw)
  To: Simon Hausmann; +Cc: Git

On Wed, Sep 19, 2007 at 08:19:11AM +0200, Simon Hausmann wrote:

>> An additional problem:
>>
>>    - git-p4 doesn't preserve the execute permission bit from Perforce.
>
>Hmm, can you paste the output of
>
>	p4 fstat //path/in/depot/to/file/that/is/imported/incorrectly
>
>? I'm interested in the type of the file that p4 reports.

   headType kxtext

so the problem is that the git-p4 is only looking for an 'x' at the start.
According to 'p4 help filetypes', we need to use execute for any of:

   cxtext, kxtext, uxbinary,  and the others that start with 'x'.

I think it would be sufficient to check the first or second character for
an 'x'.  I'll make a change and give it a try later today.

David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19 17:12     ` David Brown
@ 2007-09-19 18:23       ` Reece Dunn
  2007-09-19 18:25         ` David Brown
  0 siblings, 1 reply; 19+ messages in thread
From: Reece Dunn @ 2007-09-19 18:23 UTC (permalink / raw)
  To: Simon Hausmann, Git

On 19/09/2007, David Brown <git@davidb.org> wrote:
> On Wed, Sep 19, 2007 at 08:19:11AM +0200, Simon Hausmann wrote:
>
> >> An additional problem:
> >>
> >>    - git-p4 doesn't preserve the execute permission bit from Perforce.
> >
> >Hmm, can you paste the output of
> >
> >       p4 fstat //path/in/depot/to/file/that/is/imported/incorrectly
> >
> >? I'm interested in the type of the file that p4 reports.
>
>    headType kxtext
>
> so the problem is that the git-p4 is only looking for an 'x' at the start.
> According to 'p4 help filetypes', we need to use execute for any of:
>
>    cxtext, kxtext, uxbinary,  and the others that start with 'x'.
>
> I think it would be sufficient to check the first or second character for
> an 'x'.  I'll make a change and give it a try later today.

These are the old file types. If you read the output of `p4 help
filetypes`, the new way of specifying this is with file type
modifiers. Therefore, you also have things like text+x.

- Reece

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19 18:23       ` Reece Dunn
@ 2007-09-19 18:25         ` David Brown
  2007-09-19 18:56           ` Reece Dunn
  0 siblings, 1 reply; 19+ messages in thread
From: David Brown @ 2007-09-19 18:25 UTC (permalink / raw)
  To: Reece Dunn; +Cc: Simon Hausmann, Git

On Wed, Sep 19, 2007 at 07:23:41PM +0100, Reece Dunn wrote:

>> I think it would be sufficient to check the first or second character for
>> an 'x'.  I'll make a change and give it a try later today.
>
>These are the old file types. If you read the output of `p4 help
>filetypes`, the new way of specifying this is with file type
>modifiers. Therefore, you also have things like text+x.

So my patch I just sent may not be sufficient.  Thing is, we set the file
type as 'text+x' and it comes back as xtext, so I'm not sure if P4 ever
gives out text+x or if that is just available as a new way of specifying
them.

David

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19 18:25         ` David Brown
@ 2007-09-19 18:56           ` Reece Dunn
  0 siblings, 0 replies; 19+ messages in thread
From: Reece Dunn @ 2007-09-19 18:56 UTC (permalink / raw)
  To: Reece Dunn, Simon Hausmann, Git

On 19/09/2007, David Brown <git@davidb.org> wrote:
> On Wed, Sep 19, 2007 at 07:23:41PM +0100, Reece Dunn wrote:
>
> >> I think it would be sufficient to check the first or second character for
> >> an 'x'.  I'll make a change and give it a try later today.
> >
> >These are the old file types. If you read the output of `p4 help
> >filetypes`, the new way of specifying this is with file type
> >modifiers. Therefore, you also have things like text+x.
>
> So my patch I just sent may not be sufficient.  Thing is, we set the file
> type as 'text+x' and it comes back as xtext, so I'm not sure if P4 ever
> gives out text+x or if that is just available as a new way of specifying
> them.

I'm not sure. The Perforce help says that xtext and its variants are
there for backward compatibility. If you are running an older server
with a new client, or the other way around, they may be doing a map
from text+x to xtext so that the old version can work properly. This
is just speculation, though.

I don't know enough to say what Perforce is doing. I find it strange
that it is reporting xtext, when you specified text+x.

Have you tried a combination that is not supported? Is xunicode
supported, in which case you could try unicode+x (if you have a file
that you can experiment with)?

- Reece

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-19  0:23       ` Sam Vilain
@ 2007-09-19 21:20         ` Reece Dunn
  0 siblings, 0 replies; 19+ messages in thread
From: Reece Dunn @ 2007-09-19 21:20 UTC (permalink / raw)
  To: Sam Vilain; +Cc: Git

On 19/09/2007, Sam Vilain <sam@vilain.net> wrote:
> Reece Dunn wrote:
> > There is no need to create yet another Perforce importing tool, git-p4
> > works well in most cases. If we focus on improving git-p4, extending
> > it to support the functionality mentioned here, fix the issues that
> > there are with it, then that will be more beneficial to the community
> > as they will not have to learn another tool with a different set of
> > bugs and issues.
>
> I like my approach; it's clean and I think shows a tasteful level of
> distrust towards the sanity and integrity of the data held by Perforce.
>  Actually it really helped me understand what was really going on;
> because the information as displayed by for instance "p4 integrate" is
> a lot more confusing than the underlying tables (IMHO).

I agree. What I wasn't clear about in that paragraph, but had eluded
to in other comments in that email, is that having both git-p4 and
git-p4raw is a good thing as they operate on two differing use cases.
What I was referring to there is to have another equivalent of git-p4
that interfaced using the p4 client.

- Reece

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: State of Perforce importing.
  2007-09-18 15:49   ` David Brown
  2007-09-18 17:53     ` Reece Dunn
@ 2007-09-20  6:12     ` Dmitry Kakurin
  1 sibling, 0 replies; 19+ messages in thread
From: Dmitry Kakurin @ 2007-09-20  6:12 UTC (permalink / raw)
  To: David Brown; +Cc: Git

[-- Attachment #1: Type: text/plain, Size: 3005 bytes --]

I had to import quite a big Perforce depot too. And after some struggle it 
went fine.
For case mismatch: 
http://kb.perforce.com/AdminTasks/SuperuserTasks/CrossPlatfor..erMigration, 
bullet 10. It requires server access.
For a@b and for excluding one p4 path during migration you could use my 
quick-and-dirty fix (attached).
You can easily extend it to include and exclude multiple paths. Then it 
becomes as flexible as p4 client mapping.

- Dmitry
----- Original Message ----- 
From: "David Brown" <git@davidb.org>
Newsgroups: gmane.comp.version-control.git
To: "Sam Vilain" <sam@vilain.net>
Cc: "Git" <git@vger.kernel.org>
Sent: Tuesday, 18 September 2007 8:49
Subject: Re: State of Perforce importing.


> On Tue, Sep 18, 2007 at 07:27:13PM +1200, Sam Vilain wrote:
>
>>I'm pretty close to giving a newer one a spin, that actually imports
>>from the raw perforce back-end files without needing the perforce
>>server.  I am hoping that this should give a very clean import and will
>>be very fast and efficient, sending files that share ancestry to gfi in
>>sequence so that the on-the-fly delta system works.
>
> Unfortunately, this isn't something I'm going to be able to use.  The
> Perforce server will remain live, and resides on a machine I don't have
> access to.
>
>>It could possibly be adapted to use the p4 client (though I'd expect
>>that to be relatively slow per-revision), and possibly be extended to be
>>bidirectional as all of the upstream change number information is
>>recorded, a la git-svn.
>
> I was able to get 'git-p4' to work a lot better by using @all, but it 
> still
> has some problems, at least bad interactions with P4.
>
>   - It doesn't use any client spec.  Our P4 server space is a complete
>     mismash and has to be fixed up to get a sane directory layout.  For
>     example, some revisions have hundred-MB tar files sitting in the root
>     directory and I don't want that in the repo.  I also need to exclude
>     directories, and in some cases completely rearrange the directory
>     layout.
>
>   - Our P4 server is set to be case insensitive.  'git-p4' ignores paths
>     that come back from the server that are specified using a different
>     case.  Unfortunately, this means that a handful of files just get
>     randomly dropped from each revision.
>
>     I tried importing a client path instead of a depot path, but the names
>     that come back from 'p4 files' are depot based so none ever match.  I
>     end up with a nice revision history of entirely empty trees.
>
> I'm probably going to end up writing an importer that uses an actual 
> client
> workspace to let Perforce do the client mapping.  I'm also going to have 
> to
> put some work into some code to clean up the log messages, since most of
> our changes have as a first line "New Features:", which makes for a rather
> uninformative shortlog.
>
> But, I did learn about 'p4 -G' from git-p4 so that will help in getting
> information from the repository.
>
> Thanks,
> David 

[-- Attachment #2: 0001-git-p4-Added-exclude-option-and-use-P4-client-nam.patch --]
[-- Type: application/octet-stream, Size: 2664 bytes --]

>From a089b02239c3bc310956964d61075b093d26549f Mon Sep 17 00:00:00 2001
From: Dmitry Kakurin <Dmitry.Kakurin@gmail.com>
Date: Sun, 9 Sep 2007 13:58:12 -0700
Subject: [PATCH] git-p4: Added --exclude option and use P4 client name in commits

---
 contrib/fast-import/git-p4 |   19 +++++++++++++++----
 1 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index adaaae6..337854f 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -769,6 +769,7 @@ class P4Sync(Command):
         self.keepRepoPath = False
         self.depotPaths = None
         self.p4BranchesInGit = []
+        self.cloneExclude = None
 
         if gitConfig("git-p4.syncFromOrigin") == "false":
             self.syncWithOrigin = False
@@ -779,8 +780,12 @@ class P4Sync(Command):
         while commit.has_key("depotFile%s" % fnum):
             path =  commit["depotFile%s" % fnum]
 
-            found = [p for p in self.depotPaths
-                     if path.startswith (p)]
+            if self.cloneExclude and path.startswith( self.cloneExclude ):
+                found = False
+            else:
+                found = [p for p in self.depotPaths
+                         if path.startswith (p)]
+
             if not found:
                 fnum = fnum + 1
                 continue
@@ -905,6 +910,8 @@ class P4Sync(Command):
         else:
             committer = "%s <a@b> %s %s" % (author, epoch, self.tz)
 
+        committer = "%s <%s@%s> %s %s" % (author, details["user"], details["client"], epoch, self.tz)
+
         self.gitStream.write("committer %s\n" % committer)
 
         self.gitStream.write("data <<EOT\n")
@@ -1540,10 +1547,14 @@ class P4Clone(P4Sync):
         P4Sync.__init__(self)
         self.description = "Creates a new git repository and imports from Perforce into it"
         self.usage = "usage: %prog [options] //depot/path[@revRange]"
-        self.options.append(
+        self.options += [
             optparse.make_option("--destination", dest="cloneDestination",
                                  action='store', default=None,
-                                 help="where to leave result of the clone"))
+                                 help="where to leave result of the clone"),
+            optparse.make_option("--exclude", dest="cloneExclude",
+                                 action='store', default=None,
+                                 help="exclude depot path")
+        ]
         self.cloneDestination = None
         self.needsGit = False
 
-- 
1.5.3.mingw.1.1.g01e3a1


^ permalink raw reply related	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2007-09-20  6:13 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-09-17 19:30 State of Perforce importing David Brown
2007-09-18  6:58 ` Simon Hausmann
2007-09-18  7:27 ` Sam Vilain
2007-09-18 15:49   ` David Brown
2007-09-18 17:53     ` Reece Dunn
2007-09-18 23:19       ` David Brown
2007-09-19  0:20         ` Sam Vilain
2007-09-19  0:26           ` David Brown
2007-09-19  0:23       ` Sam Vilain
2007-09-19 21:20         ` Reece Dunn
2007-09-20  6:12     ` Dmitry Kakurin
2007-09-18 23:37 ` David Brown
2007-09-19  0:23   ` Sam Vilain
2007-09-19  0:27     ` David Brown
2007-09-19  6:19   ` Simon Hausmann
2007-09-19 17:12     ` David Brown
2007-09-19 18:23       ` Reece Dunn
2007-09-19 18:25         ` David Brown
2007-09-19 18:56           ` Reece Dunn

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).