Inconsistent Behavior using 'Reference'

Inconsistent Behavior using 'Reference'

[Bret Barkelew]

The repo/workspace (not the cache, the code we’re going to build) that
we’re trying to initialize uses several submodules. We’ve notice that
if we use ‘clone’ first on the parent repository, then call ‘git
submodule update --init --recursive --reference <path>’ inside the
parent repository, the same path is passed to all child and nested
child repositories.

However, if we call ‘git clone --recurse-submodules --reference
<path>’ and try to clone the parent and initialize submodules in one
step, Git tries to append the submodule relative path (relative to the
parent) to each of the recursive calls, and since the reference repo
is bare, this fails.

CRITICAL - Cloning repo: https://github.com/Microsoft/mu_tiano_plus.git
INFO - Cmd to run is: git clone --recurse-submodules --reference
C:\src2\mu4\mu_basecore https://github.com/Microsoft/mu_tiano_plus.git
C:\src2\mu4\mu_basecore\Common\TIANO
INFO - ------------------------------------------------
INFO - --------------Cmd Output Starting---------------
INFO - ------------------------------------------------
INFO - Cloning into 'C:\src2\mu4\mu_basecore\Common\TIANO'...
Checking out files: 100% (3858/3858), done.
INFO - Submodule 'CryptoPkg/Library/OpensslLib/openssl'
(https://github.com/openssl/openssl) registered for path
'CryptoPkg/Library/OpensslLib/openssl'
INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
add alternate: path
'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
does not exist
INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl'. Retry scheduled
INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
add alternate: path
'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
does not exist
INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl' a second
time, aborting

As you can see, the parent path is
‘'C:\src2\mu4\mu_basecore\Common\TIANO’, but when clone initializes
the submodule, it updates the ‘C:\src2\mu4\mu_basecore’ reference to
‘C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/’,
as though the reference were a full repo and it was checking for the
submodule repo within the ‘.git’ directory.

If we do this same thing using a ‘clone’ first, and ‘submodule update’
second, the same ‘C:\src2\mu4\mu_basecore’ reference is passed to all
submodules (AND nested submodules).

Thoughts? Are these both expected behaviors? Will they be consistent
in future versions of git?

Thanks!

- Bret Barkelew

Re: Inconsistent Behavior using 'Reference'

[Bret Barkelew]

Bump.

We really do need to figure this out. Anyone familiar with the
behavior of “reference”?

Thanks!

On Tue, Jan 15, 2019 at 2:24 PM Bret Barkelew <bret@corthon.com> wrote:
>
> The repo/workspace (not the cache, the code we’re going to build) that
> we’re trying to initialize uses several submodules. We’ve notice that
> if we use ‘clone’ first on the parent repository, then call ‘git
> submodule update --init --recursive --reference <path>’ inside the
> parent repository, the same path is passed to all child and nested
> child repositories.
>
> However, if we call ‘git clone --recurse-submodules --reference
> <path>’ and try to clone the parent and initialize submodules in one
> step, Git tries to append the submodule relative path (relative to the
> parent) to each of the recursive calls, and since the reference repo
> is bare, this fails.
>
> CRITICAL - Cloning repo: https://github.com/Microsoft/mu_tiano_plus.git
> INFO - Cmd to run is: git clone --recurse-submodules --reference
> C:\src2\mu4\mu_basecore https://github.com/Microsoft/mu_tiano_plus.git
> C:\src2\mu4\mu_basecore\Common\TIANO
> INFO - ------------------------------------------------
> INFO - --------------Cmd Output Starting---------------
> INFO - ------------------------------------------------
> INFO - Cloning into 'C:\src2\mu4\mu_basecore\Common\TIANO'...
> Checking out files: 100% (3858/3858), done.
> INFO - Submodule 'CryptoPkg/Library/OpensslLib/openssl'
> (https://github.com/openssl/openssl) registered for path
> 'CryptoPkg/Library/OpensslLib/openssl'
> INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
> add alternate: path
> 'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
> does not exist
> INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl'. Retry scheduled
> INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
> add alternate: path
> 'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
> does not exist
> INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl' a second
> time, aborting
>
> As you can see, the parent path is
> ‘'C:\src2\mu4\mu_basecore\Common\TIANO’, but when clone initializes
> the submodule, it updates the ‘C:\src2\mu4\mu_basecore’ reference to
> ‘C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/’,
> as though the reference were a full repo and it was checking for the
> submodule repo within the ‘.git’ directory.
>
> If we do this same thing using a ‘clone’ first, and ‘submodule update’
> second, the same ‘C:\src2\mu4\mu_basecore’ reference is passed to all
> submodules (AND nested submodules).
>
> Thoughts? Are these both expected behaviors? Will they be consistent
> in future versions of git?

If we were to converge on one, we would prefer the single path rather
than the inferred path.

>
> Thanks!
>
> - Bret Barkelew

Re: Inconsistent Behavior using 'Reference'

[Bret Barkelew]

Bump x 2

Still seeking anyone that understands the logic behind how "reference"
works across multiple commands.

Thanks!
- Bret

On Sat, Jan 19, 2019 at 10:38 AM Bret Barkelew <bret@corthon.com> wrote:
>
> Bump.
>
> We really do need to figure this out. Anyone familiar with the
> behavior of “reference”?
>
> Thanks!
>
> On Tue, Jan 15, 2019 at 2:24 PM Bret Barkelew <bret@corthon.com> wrote:
> >
> > The repo/workspace (not the cache, the code we’re going to build) that
> > we’re trying to initialize uses several submodules. We’ve notice that
> > if we use ‘clone’ first on the parent repository, then call ‘git
> > submodule update --init --recursive --reference <path>’ inside the
> > parent repository, the same path is passed to all child and nested
> > child repositories.
> >
> > However, if we call ‘git clone --recurse-submodules --reference
> > <path>’ and try to clone the parent and initialize submodules in one
> > step, Git tries to append the submodule relative path (relative to the
> > parent) to each of the recursive calls, and since the reference repo
> > is bare, this fails.
> >
> > CRITICAL - Cloning repo: https://github.com/Microsoft/mu_tiano_plus.git
> > INFO - Cmd to run is: git clone --recurse-submodules --reference
> > C:\src2\mu4\mu_basecore https://github.com/Microsoft/mu_tiano_plus.git
> > C:\src2\mu4\mu_basecore\Common\TIANO
> > INFO - ------------------------------------------------
> > INFO - --------------Cmd Output Starting---------------
> > INFO - ------------------------------------------------
> > INFO - Cloning into 'C:\src2\mu4\mu_basecore\Common\TIANO'...
> > Checking out files: 100% (3858/3858), done.
> > INFO - Submodule 'CryptoPkg/Library/OpensslLib/openssl'
> > (https://github.com/openssl/openssl) registered for path
> > 'CryptoPkg/Library/OpensslLib/openssl'
> > INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
> > add alternate: path
> > 'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
> > does not exist
> > INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl'. Retry scheduled
> > INFO - fatal: submodule 'CryptoPkg/Library/OpensslLib/openssl' cannot
> > add alternate: path
> > 'C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/'
> > does not exist
> > INFO - Failed to clone 'CryptoPkg/Library/OpensslLib/openssl' a second
> > time, aborting
> >
> > As you can see, the parent path is
> > ‘'C:\src2\mu4\mu_basecore\Common\TIANO’, but when clone initializes
> > the submodule, it updates the ‘C:\src2\mu4\mu_basecore’ reference to
> > ‘C:/src2/mu4/mu_basecore/.git/modules/CryptoPkg/Library/OpensslLib/openssl/’,
> > as though the reference were a full repo and it was checking for the
> > submodule repo within the ‘.git’ directory.
> >
> > If we do this same thing using a ‘clone’ first, and ‘submodule update’
> > second, the same ‘C:\src2\mu4\mu_basecore’ reference is passed to all
> > submodules (AND nested submodules).
> >
> > Thoughts? Are these both expected behaviors? Will they be consistent
> > in future versions of git?
>
> If we were to converge on one, we would prefer the single path rather
> than the inferred path.
>
> >
> > Thanks!
> >
> > - Bret Barkelew