"git submodule" vs "git subtree" vs "repo" vs "git subdir" ... ?

"git submodule" vs "git subtree" vs "repo" vs "git subdir" ... ?

[Robert P. J. Day]

  looking for general opinions ... i am (frighteningly :-) teaching a
git course later this week, and one of the topics on the list is git
submodules, which was specifically requested by the client as their
idea of how to start incorporating child repos in new projects.

  however, given the number of articles written about the drawbacks
with submodules, i wanted to throw in a section about "git subtree" as
well, even though (as discussed earlier) there is some minor dispute
as to whether "git subtree" is part of "core" git, but i'm not going
to let that stop me.

  going even beyond that, there is also google's "repo" command, which
i seem to see more and more often, like here for automotive grade
linux:

  https://wiki.automotivelinux.org/agl-distro/source-code

and it would be a shame to at least not mention that as yet another
possibility.

  and then there are unofficial, hand-rolled solutions, like
"git-subdir":

  https://github.com/andreyvit/git-subdir

given that the client does not appear to be wedded to any particular
solution yet, i'm open to recommendations or pointers to online
comparisons that i'll collect and post on a single wiki page, and they
can peruse the comparisons at their leisure.

  so ... thoughts? no need to be verbose, just links to decent online
discussions/comparisons would be massively useful.

rday

p.s. oh, pointers to well-designed usage of any of the above would be
handy as well. as i mentioned, for "repo", there's AGL. for
submodules, i might use boost:

  https://github.com/boostorg/boost/wiki/Getting-Started

and so on. thank you kindly.

Re: "git submodule" vs "git subtree" vs "repo" vs "git subdir" ... ?

[Nicolas Morey-Chaisemartin]

Le 13/02/2018 à 14:06, Robert P. J. Day a écrit :
>   looking for general opinions ... i am (frighteningly :-) teaching a
> git course later this week, and one of the topics on the list is git
> submodules, which was specifically requested by the client as their
> idea of how to start incorporating child repos in new projects.
>
>   however, given the number of articles written about the drawbacks
> with submodules, i wanted to throw in a section about "git subtree" as
> well, even though (as discussed earlier) there is some minor dispute
> as to whether "git subtree" is part of "core" git, but i'm not going
> to let that stop me.

I've dealt with a large number of submodules and subtrees in the last years so here are my 2 cents.
The first question to ask is the most important one:
Do the repos really need to be split at all.

In a previous life, we started with one repo per "small projects", each containing a couple of binaries or libs.
Everything was put together through submodules.
After way too long to admit, we realized that it was just not worth it. We had to deal with so many corner cases in CI and other things for a no real gain.
The reality is git works perfectly with bigger projects and smart enough that people working on two different "subprojects" don't create any conflict.
So we started merging a lot of it back into the main repo and that was it.

I see some advantages submodules have other subtrees:
- It's slightly easier to in a subproject when using submodule than with subtree (at least for git newcomers).
Once you made sure your submodule is on a branch, you just go into it and work in a standard git.
When using subtree, you have to know you're in a subtree and to your subtree push/pull to resync with someone working directly in the subproject.
At the time conflict in subtrees were sometimes a bit weird and created history full of merge.

- submodules are great when working with huge subrepo.
We have a repository than had among its submodule: linux, gcc, gdb, glibc.
This means a real lot of code/files and commits.
Subtree would work but you'd endup with a humongous repo and as expected everything is slowed down.

- submodules work well when you are moving between versions and branches of a subproject.
Let me clarify with an example:
We had a repo that had ffmpeg as a submodule with a large patch series from us.
We would regularly create a new branch in the subrepo with all our patches on top of a new ffmpeg upstream release.
And simply point the top project to this SHA1. And it worked great.
I wouldn't risk doing that kind of things with subtree. I may be wrong here (I haven't tried it to be honest) the the push/pull approach doesn't seem to fit
 the idea of moving from a SHA1 to a seemingly unrelated one.

Now that I'm working with reasonable repo sizes again, and linear history, I mostly stick with subtrees.

Nicolas

Re: "git submodule" vs "git subtree" vs "repo" vs "git subdir" ... ?

[Stefan Beller]

On Tue, Feb 13, 2018 at 5:06 AM, Robert P. J. Day <rpjday@crashcourse.ca> wrote:
>
>   looking for general opinions ... i am (frighteningly :-) teaching a
> git course later this week, and one of the topics on the list is git
> submodules, which was specifically requested by the client as their
> idea of how to start incorporating child repos in new projects.
>
>   however, given the number of articles written about the drawbacks
> with submodules, i wanted to throw in a section about "git subtree" as
> well, even though (as discussed earlier) there is some minor dispute
> as to whether "git subtree" is part of "core" git, but i'm not going
> to let that stop me.
>
>   going even beyond that, there is also google's "repo" command, which
> i seem to see more and more often, like here for automotive grade
> linux:
>
>   https://wiki.automotivelinux.org/agl-distro/source-code
>
> and it would be a shame to at least not mention that as yet another
> possibility.

Please note that Google would prefer to get rid of the repo tool.
(It was made as a stop gap solution until submodules are good enough,
i.e. have comparable UX compared to repo. But as you know stop gap
solutions hold up for quite a long time reliably. :) repo has issues by itself,
fundamental issues such as the data model, as well as minor things like
complete lack of tests)

>   and then there are unofficial, hand-rolled solutions, like
> "git-subdir":
>
>   https://github.com/andreyvit/git-subdir
>
> given that the client does not appear to be wedded to any particular
> solution yet, i'm open to recommendations or pointers to online
> comparisons that i'll collect and post on a single wiki page, and they
> can peruse the comparisons at their leisure.

There are a couple of these. I came across these recently
http://gitslave.sourceforge.net/
https://github.com/ingydotnet/git-subrepo

>   so ... thoughts? no need to be verbose, just links to decent online
> discussions/comparisons would be massively useful.
>
> rday
>
> p.s. oh, pointers to well-designed usage of any of the above would be
> handy as well. as i mentioned, for "repo", there's AGL. for
> submodules, i might use boost:

If your copy of Git is recent, look at the man pages for submodules
such as d48034551a (submodules: overhaul documentation, 2017-06-22)
or 4f73a7f124 (Doc/gitsubmodules: make some changes to improve
readability and syntax, 2018-01-14)

>   https://github.com/boostorg/boost/wiki/Getting-Started
>
> and so on. thank you kindly.

Stefan