Re: [RFC/PATCH] add update to branch support for "floating submodules"

[Phil Hord <phil.hord@gmail.com>]

I lost my grip on this thread over the holidays...

On Tue, Dec 13, 2011 at 4:09 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
> Am 13.12.2011 15:17, schrieb Phil Hord:
>> On Mon, Dec 12, 2011 at 2:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> [...]
>>> Distro package dependency tracking is a poor analogy for many reasons, but
>>> I'll only touch a few.
>> [...]
>>> Naively, one might think that two branches, branch-1.0 and branch-2.0, can
>>> be defined in the repository of L, tell somebody (like "superproject that
>>> covers all the packages in the distro") that A wants branch-1.0 and B
>>> wants branch-2.0 of L respectively, to emulate this, but if one thinks
>>> further, one would realize that it is insufficient. For one thing, it is
>>> unclear what should happen when both A and B are asked to be checked out,
>>> but more importantly, in dependency requirements on real distro packaging,
>>> the application C could say "I want v1.0 API but v1.4 is broken and not
>>> compatible with me", which won't fit on the two-branches model. A
>>> workaround to add more branches to L could be devised but any workaround
>>> cannot be a good solution that allows a random application C among 47
>>> others to dictate how the branch structure of L project should look like.
>>>
>>> Fortunately, the dependency management is a solved problem by distro
>>> package management and build systems, and they do so without using
>>> anything from submodules. There is no point reinventing these logic in git
>>> submodules and emulating poorly.
>>>
>>> The only remotely plausible analogy around distro packaging would be a
>>> superproject full of all the packages in a distro as its submodules, and
>>> records exact versions of each and every package that goes on a release
>>> CD (or DVD). In that case, you do want to have a central registry that
>>> records what exact version of each package is used to cut the CD and the
>>> mother of all modules superproject could be one way to implement it. But
>>> that is not an example of floating, but is a direct opposite.
>>>
>>> This exchange convinced me further that anybody who wishes to use
>>> "floating" is better off either by doing one or both of the following:
>>>
>>>  - using "exact" but not updating religiously, as the interdepency
>>>   requirement in their project is not strict; or
>>>
>>>  - not using submodules at all, but merely keeping these unrelated A, B, C
>>>   and L as standalone repositories next to each other in the directory
>>>   structure.
>>
>> My interdependency requirements are not so cut-and-dry.  We use
>> submodules to isolate controlled regions of code.  We may need to
>> share our project with a contractor who is allowed to see code
>> pertaining to "vendorA" but not that for "vendorB" or "VendorN".  But
>> our in-house developers want to have all the vendor code in one place
>> for convenient integration. Submodules do this nicely for us.  We can
>> give the contractor just the main modules and the VendorA modules and
>> he'll be fine.  In-house devs get all the submodules (using the
>> vendor-ALL superproject).
>>
>> But this necessarily means there is too much coupling for comfort
>> between our submodules.   For example, when an API changes in the main
>> submodule, each of the vendor submodules is affected because they each
>> implement that API in a custom method.  Some of those vendor modules
>> belong to different people.  Submodule synchronization becomes a real
>> chore.
>
> Hmm, maybe having vendor-specific branches in the superproject would
> help here. But that is hard to tell without knowing more details about
> your setup. But I suspect your vendor-ALL superproject is exactly the
> right spot to deal with these kind of problems (and if that isn't easy
> that might be a result of the difficulty of the problem you are trying
> to solve here, keeping different vendors in sync with your API ;-).
>
>> Floating would help, I think.  Instead I do this:
>>
>>   git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'
>>
>>   git submodule foreach 'git push origin topic_foo' && git push origin topic_foo
>
> This sounds to me like you would need the "--recurse-submodules" option
> implemented for "git pull" and "git push", no?

Only if I have nested submodules, but yes, we do use --recurs* in our scripts.

> And I miss to see how
> floating would help when the tips of some submodules are not ready to
> work with other submodules tips ...

By project policy, for any branch, all submodules' tips of the
same-named branch should be interoperable.  The CI server looks after
this, as much as he can.

I think of branch names as sticky notes (extra-lightweight tags,
sometimes).  We have linear history in many of our vendor submodules,
but multiple "branches" indicate where each superproject branch has
presumably finished integration.

>> But not all my developers are git-gurus yet, and they sometimes mess
>> up their ad hoc scripts or miss important changes they forgot to push
>> in one submodule or another.
>
> Sure, even though current git should help you some by showing changes
> in the submodules.

Real newbies may not even remember to use 'git status' strategically.

>>  Or worse, their pull or push fails and
>> they can't see the problem for all the noise.  So they email it to me.
>
> We circumvent that by not pulling, but fetching and merging in the
> submodule first and after that in the superproject. You have much more
> control about what is going wrong where (and can have more
> git-experienced people help with - or even do - the merges).

I do that, too, and I wish I didn't have to.  I wish I could safely
and sanely recover from a conflicted "git pull --recurse-submodules"
pull from the superproject.  That is, I wish doing so were as
straightforward as recovering from the same condition would be if all
my code were in one repository instead of in submodules.

Which is the gist -- I wish submodules did not make git more
complicated than it already is.

Phil