git projects with submodules in different sites - in txt format (:+(

git projects with submodules in different sites - in txt format (:+(

[Michele Hallak]

Hi,

I am getting out of idea about how to change the methodology we are using in order to ease our integration process... Close to despair, I am throwing the question to you...

We have 6 infrastructure repositories.
Each project is composed of 4 repositories, each one using one or two infrastructure repositories as sub-modules. (Not the same)

The infrastructure repositories are common to several projects and in the case we have to make change in the infrastructure for a specific project, we are doing it on a specific branch until properly merged.

Everything is fine (more or less) and somehow working.

Now, we have one project that will be developed in another site and with another git server physically separated from the main site.

I copied the infrastructure repositories in the new site and removed and add the sub-modules in order for them to point to the url in the separated git server.

Every 2 weeks, the remotely developed code has to be integrated back in the main site. 
My idea was to format GIT patches, integrate in the main site, tag the whole thing and ship back the integrated tagged code to the remote site.
... and now the nightmare starts:

Since the .gitmodules is different, I cannot have the same SHA and then same tag and I am never sure that the integrated code is proper.

May be there is a simple solution that I don't know about to my problem? Is there something else than GIT patches? Should I simply ship to the remote site the code as is and change the submodules each time?

Thanks a lot for trying to help me,

Michele



*********************************************************************************************** Please consider the environment before printing this email ! The information contained in this communication is proprietary to Israel Aerospace Industries Ltd. and/or third parties, may contain confidential or privileged information, and is intended only for the use of the intended addressee thereof. If you are not the intended addressee, please be aware that any use, disclosure, distribution and/or copying of this communication is strictly prohibited. If you receive this communication in error, please notify the sender immediately and delete it from your computer. Thank you. Visit us at: www.iai.co.il

Re: git projects with submodules in different sites - in txt format (:+(

[Philip Oakley]

On 02/10/2018 06:47, Michele Hallak wrote:
> Hi,
> 
> I am getting out of idea about how to change the methodology we are using in order to ease our integration process... Close to despair, I am throwing the question to you...
> 
> We have 6 infrastructure repositories [A, B, C, D, E, F ?].

> Each project [W,X,Y,Z] is composed of 4 repositories [1-4], each one using one or two infrastructure repositories as sub-modules. (Not the same)

e.g. W1-W4; with say B & D as submodules
> 
> The infrastructure repositories are common to several projects and in the case we have to make change in the infrastructure for a specific project, we are doing it on a specific branch until properly merged.

Do you also have remotes setup that provide backup and central authority 
to the projects..?
> 
> Everything is fine (more or less) and somehow working.

Good..
> 
> Now, we have one project that will be developed in another site and with another git server physically separated from the main site.

Is it networked? Internal control, external internet, sneakernet?
> 
> I copied the infrastructure repositories in the new site and removed and add the sub-modules in order for them to point to the url in the separated git server.
> 
> Every 2 weeks, the remotely developed code has to be integrated back in the main site.
> My idea was to format GIT patches, integrate in the main site, tag the whole thing and ship back the integrated tagged code to the remote site.
> ... and now the nightmare starts:
yep, you have lost the validation & verification capability of Git's 
sha1/oid and DAG.

> 
> Since the .gitmodules is different, I cannot have the same SHA and then same tag and I am never sure that the integrated code is proper.

Remotes, remotes...
> 
> May be there is a simple solution that I don't know about to my problem? Is there something else than GIT patches? Should I simply ship to the remote site the code as is and change the submodules each time?
>

I think the solution you need is `git bundle` 
https://git-scm.com/docs/git-bundle. This is designed for the case where 
you do not have the regular git transport infrastructure. Instead it 
records the expected data that would be 'on the wire', which is then 
read in at the far end. The bundle can contain excess data to ensure 
overlap between site transmissions.

You just run the projects in the same way but add the courier step for 
shipping the CD, or some password protected archive as per your security 
needs.

Everything is should be just fine (more or less) and somehow it will 
just work. ;-)

--
Philip
https://stackoverflow.com/questions/11792671/how-to-git-bundle-a-complete-repo

RE: git projects with submodules in different sites - in txt format (:+(

[Michele Hallak]

Thank you so much, I will check this GIT bundle....
Since you asked, here are my answers suffixed by MHS >>

Michele

-----Original Message-----
From: Philip Oakley [mailto:philipoakley@iee.org] 
Sent: Wednesday, October 3, 2018 12:00 AM
To: Michele Hallak; git@vger.kernel.org
Subject: Re: git projects with submodules in different sites - in txt format (:+(

On 02/10/2018 06:47, Michele Hallak wrote:
> Hi,
> 
> I am getting out of idea about how to change the methodology we are using in order to ease our integration process... Close to despair, I am throwing the question to you...
> 
> We have 6 infrastructure repositories [A, B, C, D, E, F ?].

> Each project [W,X,Y,Z] is composed of 4 repositories [1-4], each one 
> using one or two infrastructure repositories as sub-modules. (Not the 
> same)

e.g. W1-W4; with say B & D as submodules

MHS >> Exact

> 
> The infrastructure repositories are common to several projects and in the case we have to make change in the infrastructure for a specific project, we are doing it on a specific branch until properly merged.

Do you also have remotes setup that provide backup and central authority to the projects..?

MHS >> If you mean that we have a GIT server, as yes... 
> 
> Everything is fine (more or less) and somehow working.

Good..
> 
> Now, we have one project that will be developed in another site and with another git server physically separated from the main site.

Is it networked? Internal control, external internet, sneakernet?

MHS>>> Totally disconnected - Two isolated networks.
> 
> I copied the infrastructure repositories in the new site and removed and add the sub-modules in order for them to point to the url in the separated git server.
> 
> Every 2 weeks, the remotely developed code has to be integrated back in the main site.
> My idea was to format GIT patches, integrate in the main site, tag the whole thing and ship back the integrated tagged code to the remote site.
> ... and now the nightmare starts:
yep, you have lost the validation & verification capability of Git's sha1/oid and DAG.

MHS>>> It's soothing to be understood. (:+)

> 
> Since the .gitmodules is different, I cannot have the same SHA and then same tag and I am never sure that the integrated code is proper.

Remotes, remotes...
> 
> May be there is a simple solution that I don't know about to my problem? Is there something else than GIT patches? Should I simply ship to the remote site the code as is and change the submodules each time?
>

I think the solution you need is `git bundle` https://git-scm.com/docs/git-bundle. This is designed for the case where you do not have the regular git transport infrastructure. Instead it records the expected data that would be 'on the wire', which is then read in at the far end. The bundle can contain excess data to ensure overlap between site transmissions.

You just run the projects in the same way but add the courier step for shipping the CD, or some password protected archive as per your security needs.

Everything is should be just fine (more or less) and somehow it will just work. ;-)

MHS>>> I am hopeful! Thanks a lot

--
Philip
https://stackoverflow.com/questions/11792671/how-to-git-bundle-a-complete-repo

Default Profile
*********************************************************************************************** Please consider the environment before printing this email ! The information contained in this communication is proprietary to Israel Aerospace Industries Ltd. and/or third parties, may contain confidential or privileged information, and is intended only for the use of the intended addressee thereof. If you are not the intended addressee, please be aware that any use, disclosure, distribution and/or copying of this communication is strictly prohibited. If you receive this communication in error, please notify the sender immediately and delete it from your computer. Thank you. Visit us at: www.iai.co.il