Need help merging unrelated histories

Need help merging unrelated histories

[Robert Dailey]

Everything I'm going to describe is related to this repository:

https://github.com/powervr-graphics/Native_SDK

This repo has several distinct branches. None of them seem to be tied
to each other. Instead of having a `master` where they branched off
each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a
copy of previous branches with no ancestry and then continued coding
on that.

What I'm trying to do is go back and rebase all of the X.X branches
onto a new `master` branch. I started off with this:

$ git checkout --orphan master

Then I committed a `.gitattributes`. Next, I rebased the first
(oldest) release branch:

$ git rebase -i --onto master origin/3.0 --root

Then I merged:

$ git checkout master && git merge --no-ff -

Next, I did 3.1:

$ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs

Using interactive mode, Git is smart enough to detect duplicate
commits and eliminates those, even though the 2 branches do not share
a merge base. I continued doing this all the way up to rebasing 4.3,
but that's when things got tough. I see a lot of 'UD', 'UA', 'AU',
'AA' merge conflicts. These are obviously due to the fact that the
branches aren't connected. But I expected was that `-X theirs` would
always favor what's on the branch being rebased. However, it seems
this only affects modified conflicts, not adds or deletes.

I was trying to find a way to bulk-resolve these. I mean, if git sees
a file added on the left AND the right, I want the right one (theirs;
the one coming from the 4.3 branch). Even though the branches are
unrelated in terms of their history, I want the net effect of the
rebase to essentially reflect the files on 4.3 itself. If a file isn't
present on HEAD, delete it. If a file exists on REBASE_HEAD but not on
HEAD, then add it. If the same file exists on both, favor the one on
REBASE_HEAD.

But I don't see a way of doing that. I tried `git checkout --theirs .`
and `git checkout REBASE_HEAD -- .` but this doesn't work with all
conflict types.

Can anyone provide some advice on how to properly restructure this
repository to create some ancestry, as if all along a `master` existed
and all release branches were based on this in a linear fashion?

Re: Need help merging unrelated histories

[Robert Dailey]

On Fri, May 24, 2019 at 9:04 AM Robert Dailey <rcdailey.lists@gmail.com> wrote:
>
> Everything I'm going to describe is related to this repository:
>
> https://github.com/powervr-graphics/Native_SDK
>
> This repo has several distinct branches. None of them seem to be tied
> to each other. Instead of having a `master` where they branched off
> each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a
> copy of previous branches with no ancestry and then continued coding
> on that.
>
> What I'm trying to do is go back and rebase all of the X.X branches
> onto a new `master` branch. I started off with this:
>
> $ git checkout --orphan master
>
> Then I committed a `.gitattributes`. Next, I rebased the first
> (oldest) release branch:
>
> $ git rebase -i --onto master origin/3.0 --root
>
> Then I merged:
>
> $ git checkout master && git merge --no-ff -
>
> Next, I did 3.1:
>
> $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs
>
> Using interactive mode, Git is smart enough to detect duplicate
> commits and eliminates those, even though the 2 branches do not share
> a merge base. I continued doing this all the way up to rebasing 4.3,
> but that's when things got tough. I see a lot of 'UD', 'UA', 'AU',
> 'AA' merge conflicts. These are obviously due to the fact that the
> branches aren't connected. But I expected was that `-X theirs` would
> always favor what's on the branch being rebased. However, it seems
> this only affects modified conflicts, not adds or deletes.
>
> I was trying to find a way to bulk-resolve these. I mean, if git sees
> a file added on the left AND the right, I want the right one (theirs;
> the one coming from the 4.3 branch). Even though the branches are
> unrelated in terms of their history, I want the net effect of the
> rebase to essentially reflect the files on 4.3 itself. If a file isn't
> present on HEAD, delete it. If a file exists on REBASE_HEAD but not on
> HEAD, then add it. If the same file exists on both, favor the one on
> REBASE_HEAD.
>
> But I don't see a way of doing that. I tried `git checkout --theirs .`
> and `git checkout REBASE_HEAD -- .` but this doesn't work with all
> conflict types.
>
> Can anyone provide some advice on how to properly restructure this
> repository to create some ancestry, as if all along a `master` existed
> and all release branches were based on this in a linear fashion?

By the way, everything I've done so far (up to 4.2) is located here:

https://github.com/rcdailey/Native_SDK

There is a `master` branch in my fork of that repo. I am not sure if
I've done it correctly so far, or if how I've done this up to 4.2
explains why 4.3 is so hard.

Re: Need help merging unrelated histories

[Ævar Arnfjörð Bjarmason]

On Fri, May 24 2019, Robert Dailey wrote:

> Everything I'm going to describe is related to this repository:
>
> https://github.com/powervr-graphics/Native_SDK
>
> This repo has several distinct branches. None of them seem to be tied
> to each other. Instead of having a `master` where they branched off
> each of their releases (e.g. 3.1, 3.2, 4.0), it looks like they made a
> copy of previous branches with no ancestry and then continued coding
> on that.
>
> What I'm trying to do is go back and rebase all of the X.X branches
> onto a new `master` branch. I started off with this:
>
> $ git checkout --orphan master
>
> Then I committed a `.gitattributes`. Next, I rebased the first
> (oldest) release branch:
>
> $ git rebase -i --onto master origin/3.0 --root
>
> Then I merged:
>
> $ git checkout master && git merge --no-ff -
>
> Next, I did 3.1:
>
> $ git rebase -i --onto master origin/3.0 origin/3.1 -X theirs
>
> Using interactive mode, Git is smart enough to detect duplicate
> commits and eliminates those, even though the 2 branches do not share
> a merge base. I continued doing this all the way up to rebasing 4.3,
> but that's when things got tough. I see a lot of 'UD', 'UA', 'AU',
> 'AA' merge conflicts. These are obviously due to the fact that the
> branches aren't connected. But I expected was that `-X theirs` would
> always favor what's on the branch being rebased. However, it seems
> this only affects modified conflicts, not adds or deletes.
>
> I was trying to find a way to bulk-resolve these. I mean, if git sees
> a file added on the left AND the right, I want the right one (theirs;
> the one coming from the 4.3 branch). Even though the branches are
> unrelated in terms of their history, I want the net effect of the
> rebase to essentially reflect the files on 4.3 itself. If a file isn't
> present on HEAD, delete it. If a file exists on REBASE_HEAD but not on
> HEAD, then add it. If the same file exists on both, favor the one on
> REBASE_HEAD.
>
> But I don't see a way of doing that. I tried `git checkout --theirs .`
> and `git checkout REBASE_HEAD -- .` but this doesn't work with all
> conflict types.
>
> Can anyone provide some advice on how to properly restructure this
> repository to create some ancestry, as if all along a `master` existed
> and all release branches were based on this in a linear fashion?

We don't have a merge strategy to do this, but should. I had a WIP patch
for this here that I haven't picked up:
https://public-inbox.org/git/87sh7sdtc1.fsf@evledraar.gmail.com/

But you can emulate the same thing with "read-tree" and
"commit". I.e. manually craft a commit with plumbing that diverges from
the "master" branch, then instead of stock "rebase" write some loop
where you keep using "read-tree" to read what you want to stage into the
index from the existing branched commits, then the equivalent of "git
commit -c" etc. to commit it.

Re: Need help merging unrelated histories

[Andreas Schwab]

On Mai 24 2019, Robert Dailey <rcdailey.lists@gmail.com> wrote:

> Can anyone provide some advice on how to properly restructure this
> repository to create some ancestry, as if all along a `master` existed
> and all release branches were based on this in a linear fashion?

How about using git replace --graft, then git filter-branch to make it
permanent?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: Need help merging unrelated histories

[Robert Dailey]

On Fri, May 24, 2019 at 12:11 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
>
> On Mai 24 2019, Robert Dailey <rcdailey.lists@gmail.com> wrote:
>
> > Can anyone provide some advice on how to properly restructure this
> > repository to create some ancestry, as if all along a `master` existed
> > and all release branches were based on this in a linear fashion?
>
> How about using git replace --graft, then git filter-branch to make it
> permanent?

I unfortunately have little-to-no experience with low level plumbing
commands. Could you provide an example of a series of commands to run?
It will help me to figure out how to do it on my own for my specific
use case. Thanks in advance.

Re: Need help merging unrelated histories

[Robert Dailey]

On Tue, May 28, 2019 at 8:35 AM Robert Dailey <rcdailey.lists@gmail.com> wrote:
>
> On Fri, May 24, 2019 at 12:11 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
> >
> > On Mai 24 2019, Robert Dailey <rcdailey.lists@gmail.com> wrote:
> >
> > > Can anyone provide some advice on how to properly restructure this
> > > repository to create some ancestry, as if all along a `master` existed
> > > and all release branches were based on this in a linear fashion?
> >
> > How about using git replace --graft, then git filter-branch to make it
> > permanent?
>
> I unfortunately have little-to-no experience with low level plumbing
> commands. Could you provide an example of a series of commands to run?
> It will help me to figure out how to do it on my own for my specific
> use case. Thanks in advance.

Toyed around with it a bit, and tried with the script below. There
were no errors, but I noticed that it erased my `.gitattributes` file
on master that I committed before performing the graft. Why did it
remove the file? I assume I'm not doing this correctly. Please advise.
Thank you.

```sh
#!/usr/bin/env bash
set -ex

if [[ ! -d Native_SDK.git ]]; then
    git clone --bare git@github.com:powervr-graphics/Native_SDK.git
else
    cd Native_SDK.git
    git fetch
    cd -
fi

rm -rf test_repo
git clone Native_SDK.git test_repo
cd test_repo

git checkout -f --orphan new_master
echo '* text=auto' > .gitattributes
git add .gitattributes
git commit -m 'Add gitattributes file'

c=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1)
git replace --graft $c new_master
git filter-branch HEAD..origin/3.4
git merge --no-edit --no-ff origin/3.4
```

Re: Need help merging unrelated histories

[Philip Oakley]

Hi Robert,
A few comments to look at, which may help.
On 28/05/2019 19:26, Robert Dailey wrote:
> On Tue, May 28, 2019 at 8:35 AM Robert Dailey <rcdailey.lists@gmail.com> wrote:
>> On Fri, May 24, 2019 at 12:11 PM Andreas Schwab <schwab@linux-m68k.org> wrote:
>>> On Mai 24 2019, Robert Dailey <rcdailey.lists@gmail.com> wrote:
>>>
>>>> Can anyone provide some advice on how to properly restructure this
>>>> repository to create some ancestry, as if all along a `master` existed
>>>> and all release branches were based on this in a linear fashion?
>>> How about using git replace --graft, then git filter-branch to make it
>>> permanent?
>> I unfortunately have little-to-no experience with low level plumbing
>> commands. Could you provide an example of a series of commands to run?
>> It will help me to figure out how to do it on my own for my specific
>> use case. Thanks in advance.
> Toyed around with it a bit, and tried with the script below. There
> were no errors, but I noticed that it erased my `.gitattributes` file
> on master that I committed before performing the graft. Why did it
> remove the file? I assume I'm not doing this correctly. Please advise.
> Thank you.
>
> ```sh
> #!/usr/bin/env bash
> set -ex
>
> if [[ ! -d Native_SDK.git ]]; then
>      git clone --bare git@github.com:powervr-graphics/Native_SDK.git
> else
>      cd Native_SDK.git
>      git fetch
>      cd -
> fi
>
> rm -rf test_repo
> git clone Native_SDK.git test_repo
> cd test_repo
>
> git checkout -f --orphan new_master
Check here what is in the files system and index - see the caveat in the 
man page for the --orphan start point and potential use of `git rm -rf .`
> echo '* text=auto' > .gitattributes
Maybe also add some mega comment line to really see it..
> git add .gitattributes
> git commit -m 'Add gitattributes file'
Check what is actually committed, in case you have more/less than expected.
> c=$(git log --oneline origin/3.4 | tail -1 | cut -d ' ' -f 1)
Is this the one you expected (maybe gitk viewer )
> git replace --graft $c new_master
Typing "git replace" without arguments, also lists all replace refs. - 
check what the new one is. Investigate.
> git filter-branch HEAD..origin/3.4
Hmm is this the right range for re-write (not checked, just feels as if 
it's wrong). Maybe also need a local branch name for origin/3.4, just 
for comparing.
> git merge --no-edit --no-ff origin/3.4
> ```
same again

Hopefully the random thoughts may prompt something.
Philip