Stitching histories of several repositories

Stitching histories of several repositories

[Ramkumar Ramachandra]

Hi,

I've written a tool to stitch the first-parent histories of several
git repositories.  To illustrate, consider that we have a toplevel git
repository inside which the other repositories reside.

  /.git
  /a/.git
  /b/.git

The tool pulls in the objects from /a/.git and /b/.git into /.git by
using alternates.  Then, it gets the list of commits by rev-list'ing
and emits a fast-import stream rewriting the paths of the files in a/
from / to a/ and the paths of files in b/ form / to b/ (using M
040000), taking care to have one unified first-parent line.  It turns
out that this is quite non-trivial when the individual git
repositories contain lots of merges.  To illustrate, this is an
example of how histories from two repositories with one merge commit
each would be stitched:

  *   c1f81f1 (refs/figlets/first/origin/master)
  |\
  | * 6eaf22f (refs/figlets/first-2/origin/master)
  * 15cd841

  *   1f2f408 (refs/figlets/second/origin/master)
  |\
  | * 4bd7fe3 (refs/figlets/second-2/origin/master)
  * bfa9f3c

  *   5e1d337 (refs/replay/stitch2, fig/stitch2)
  |\
  | * 23437d5 (refs/replay/1f2f40-2)
  |/
  *   4f2d70b
  |\
  | * e017fa5 (refs/replay/c1f81f-2)
  * c0251c7
  * 4036fea

I'd like to know whether the tool would be useful to a wider audience,
before I polish it and consider submitting it for inclusion in
contrib/.  I think the tool is especially useful for running bisect
and tracking bugs that occur in large projects that consist of many
git repositories.  Will a unified log showing commits in different
submodules be useful?

Ram

Re: Stitching histories of several repositories

[Robin Rosenberg]

----- Ursprungligt meddelande -----
> Hi,
> 
> I've written a tool to stitch the first-parent histories of several
> git repositories.  To illustrate, consider that we have a toplevel
> git
> repository inside which the other repositories reside.
> 
[...]
> 
> I'd like to know whether the tool would be useful to a wider
> audience,
> before I polish it and consider submitting it for inclusion in
> contrib/.  I think the tool is especially useful for running bisect
> and tracking bugs that occur in large projects that consist of many
> git repositories.  Will a unified log showing commits in different
> submodules be useful?

I think it is useful. I did something like that creating a super-repo
for performing a bisect over EGit and JGit over a period over a year.
I don't think I restricted myself to the first parent, which was probably
a mistake.

The solution isn't perfect, but provided that is well-known you can
live with that. A problem I found was that many stiches versions weren't
even compilable so one might want to have the option of matching commits
using a window of some sort to identify compilable combinations and have
the ability to use only those for bisect. I'm not sure it's practical though.

-- robin