Failure to fetch submodule with --depth=1 parameter

Failure to fetch submodule with --depth=1 parameter

[Grigory Yakushev]

$ git --version
git version 2.17.1

Repro:
$ git clone https://github.com/PX4/Firmware.git
$ cd Firmware
$ git submodule update --init --recursive --depth=1
...
error: Server does not allow request for unadvertised object
22df9475ca0d157e2db066a20f64c35906bf7f25
Fetched in submodule path 'Tools/sitl_gazebo', but it did not contain
22df9475ca0d157e2db066a20f64c35906bf7f25. Direct fetching of that
commit failed.

The same line without --depth=1 flag works fine:
$ git submodule update --init --recursive

Implication:
Docker uses the line with --depth=1, so it fails to build from remote
URLs. See https://github.com/docker/distribution/issues/2998

Re: Failure to fetch submodule with --depth=1 parameter

[Jeff King]

On Tue, Sep 03, 2019 at 05:30:02PM +0200, Grigory Yakushev wrote:

> $ git --version
> git version 2.17.1
> 
> Repro:
> $ git clone https://github.com/PX4/Firmware.git
> $ cd Firmware
> $ git submodule update --init --recursive --depth=1
> ...
> error: Server does not allow request for unadvertised object
> 22df9475ca0d157e2db066a20f64c35906bf7f25
> Fetched in submodule path 'Tools/sitl_gazebo', but it did not contain
> 22df9475ca0d157e2db066a20f64c35906bf7f25. Direct fetching of that
> commit failed.
> 
> The same line without --depth=1 flag works fine:
> $ git submodule update --init --recursive

I think what's happening is this: the super-project (PX4/Firmware)
points to a specific sha1 in the submodule (Tools/sitl_gazebo). But
there's no branch pointing at that sha1, so the client cannot ask for it
directly.

As a fallback we instead fetch HEAD from the submodule and hope that we
got the sha1 we needed. But because we pass along "--depth=1", that
fallback is not likely to work.

IMHO one obvious improvement on the client side is to have a third-level
fallback to do a full clone, and hope we get it then.

However, there may a be an easier solution. In the v2 protocol, Git does
not enforce any advertisement or reachability properties for items the
client asks for. So the client would be free to just ask for the sha1
directly, no fallback required.

Try setting:

  git config --global protocol.version 2

which seems to make it work.

-Peff

Re: Failure to fetch submodule with --depth=1 parameter

[Grigory Yakushev]

Thanks Jeff!

Completely solved the problem for me.

On Wed, Sep 4, 2019 at 5:14 AM Jeff King <peff@peff.net> wrote:
>
> On Tue, Sep 03, 2019 at 05:30:02PM +0200, Grigory Yakushev wrote:
>
> > $ git --version
> > git version 2.17.1
> >
> > Repro:
> > $ git clone https://github.com/PX4/Firmware.git
> > $ cd Firmware
> > $ git submodule update --init --recursive --depth=1
> > ...
> > error: Server does not allow request for unadvertised object
> > 22df9475ca0d157e2db066a20f64c35906bf7f25
> > Fetched in submodule path 'Tools/sitl_gazebo', but it did not contain
> > 22df9475ca0d157e2db066a20f64c35906bf7f25. Direct fetching of that
> > commit failed.
> >
> > The same line without --depth=1 flag works fine:
> > $ git submodule update --init --recursive
>
> I think what's happening is this: the super-project (PX4/Firmware)
> points to a specific sha1 in the submodule (Tools/sitl_gazebo). But
> there's no branch pointing at that sha1, so the client cannot ask for it
> directly.
>
> As a fallback we instead fetch HEAD from the submodule and hope that we
> got the sha1 we needed. But because we pass along "--depth=1", that
> fallback is not likely to work.
>
> IMHO one obvious improvement on the client side is to have a third-level
> fallback to do a full clone, and hope we get it then.
>
> However, there may a be an easier solution. In the v2 protocol, Git does
> not enforce any advertisement or reachability properties for items the
> client asks for. So the client would be free to just ask for the sha1
> directly, no fallback required.
>
> Try setting:
>
>   git config --global protocol.version 2
>
> which seems to make it work.
>
> -Peff