Re: git-p4: Clone p4 path with bidirectional integrations

[Aaron Miller <aaronkmiller@gmail.com>]

Hi Andrey,

Thanks so much for this detailed response, I really appreciate it.

> First of all, git-p4 should normally take only one direction from bidirectional integrations on its own.
> Do you see "p4 branch <branchABC> defines a mapping from <path1> to <path2>, but there exists another mapping from <path2> to <path1> already!"?
> If you do, it means that git-p4 will ignore <branchABC> mapping.

I was afraid that was the case!

> Also, just FYI, as far as I know, git-p4 doesn't create "merge" commits,
> so bidirectional integrations won't look different from ordinary commits in git commit graph.

Ah, I didn't realize that, thank you. Perhaps I should just sync each
branch separately then, ignoring branch mappings entirely and be done
with it.

I was hoping to generate a commit graph that properly represents
integrations as merge commits because our Perforce branches are quite
large. Storing diffs of integrations rather than discrete commits
would result in a much smaller Git repository for us. There are
*quite* a lot of integration commits.

I wonder how hard it would be to modify git-p4 to use merge commits? I
will take a look at the source. I'm somewhat surprised that's not the
case to begin with though? Maybe someone else can chime in on why
merge commits aren't used.

> Uh-oh, 5M commits!
> But given that it fails at 25% instead of 1%,
> you've got some luck. :)

Hehe :) Fortunately I don't have to migrate *all* of those commits,
but still a fair number.

> This might not be just because of bidirectional integrations per se.

I should have mentioned - there were no P4 branch mappings defined for
the depot paths in the test case I shared (maybe that's obvious).

> This error may happen if, say, there's a P4 branch mapping from staging to master,
> but master was actually created before staging.
> git-p4 tries to find a "parent branch" for master, but it doesn't exist yet,
> so git-p4 fails in an ugly way.

Yeah this is exactly correct, I've tested precisely this scenario.
Actually that's what two of the real branches I am trying to migrate
look like.

> One way to filter out troublesome P4 branch mappings is to set git-p4.branchUser to a particular user.
> But most likely, this won't help you because different people created different branch mappings over time.

I was thinking I could set git-p4.branchUser to a user that hasn't
created any branches at all and then define branch mappings I care
about in git-p4.branchList.

But pretty much all of our branches either have bidirectional
integrations or a situation like you described above where branchA was
created before branchB, and branchB integrates into branchA.

> Unfortunately, there's _no_ git-p4.branchRegexp config option,
> but it's fairly straightforward to implement -- patches welcome! ;)
> (getBranchMapping() needs to apply a regex to branch names before doing anything serious with them)

That would be a neat option! It's not really filtering the branch
mappings which is my issue though. The number of branches is small
enough that I can manage the mappings manually.

> Note, that you can't have master:staging together with staging:master,
> otherwise you'll likely run into the same problem as before.

Yeah this is really the crux of my issue.

> This might be simple or quite tedious depending on the history and branching strategies of your repositories.
> It may be as easy as just dropping some p4 branch mappings.

I'm thinking if I'm not getting merge commits anyway I may as well
just forget about the branch mappings entirely.

> However, one of the repositories I had to deal with had almost random branching strategy (with most integrations done without predefined branch mappings), so I had to spend quite some time to trace the history and figure out which branches make the most sense in git.

Fortunately our branching strategy was *mostly* well-defined and we do
have Perforce mappings defined between our branches. However most
integrations didn't actually use the branch mappings, just manually
specified depot paths. But they did fall under the branch mapping
scopes.

> so for repositories with simple/short history,

Unfortunately our repo history is anything but short or simple :(

> I recreated those merge commits manually (well, in a bash script) using `git replace --graft <commit> <parent1> <parent2>` followed by `git filter-branch --tag-name-filter cat -- --all` to make grafts permanent.
>
> (`git filter-branch` is only needed once in the very end after all `git replace` manipulations are done)
> It's perhaps better to teach git-p4 to produce merge commits, but a bash script was a low-tech low-risk option for me.
>
> Also, beware that git-p4 doesn't handle branch-into-non-empty directory properly.
> If I remember correctly, something like
> `p4 copy //depot/branchA/... //depot/branchB/... ; p4 submit; p4 copy //depot/branchC/... //depot/branchB/...; p4 submit`
> will result in branchB having _both_ branchA and branchC contents in git.
> `git filter-branch` or `git rebase` are your friends to workaround this.
> (or better fix git-p4, of course)

These are great tips, thanks! Maybe what I will do is sync each branch
separately with no branch mappings, then use this technique to create
merge commits for the initial branch creation commits only and not
worry about any other integrations.

-Aaron