git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
* Can we make git clone --recurse-submodules --shallow-submodules work for commits that are not on tags or branches
@ 2018-02-14  5:06 Ciro Santilli
  2018-02-14 19:18 ` Stefan Beller
  0 siblings, 1 reply; 2+ messages in thread
From: Ciro Santilli @ 2018-02-14  5:06 UTC (permalink / raw)
  To: git

I have a git repo with large submodules, notably the Linux kernel, and
shallow clone saves a lot of time.

However, QEMU is also a submodule, which I don't control, and QEMU has
submodules which don't point to commits that are not on any tag or
branch, so:

 git clone --recurse-submodules --shallow-submodules git://git.qemu.org/qemu.git

currently fails with:

error: Server does not allow request for unadvertised object
22ead3e0bfdb87516656453336160e0a37b066bf
Fetched in submodule path 'capstone', but it did not contain
22ead3e0bfdb87516656453336160e0a37b066bf. Direct fetching of that
commit failed.

on git 2.16.1.

Furthermore, I reproduce this locally with direct filesystem clones:
https://github.com/cirosantilli/test-git-web-interface/blob/15335d3002a3e64fc5756b69fb832a733aa63fb9/shallow-submodule.sh#L158
and on GitHub, so I'm guessing it is not just the settings for a
specific server?

Would it be possible to make that work, or are there fundamental
reasons why it is not possible?

Here is my use case repo, at the point of the ugly workaround I'm
having to do: https://github.com/cirosantilli/linux-kernel-module-cheat/blob/a14c95346cfd9d2e7b2e475b0981aa71d819c20b/configure#L23

Some more context:
https://stackoverflow.com/questions/2144406/git-shallow-submodules/47374702#47374702

This would make some embedded people happy.

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Can we make git clone --recurse-submodules --shallow-submodules work for commits that are not on tags or branches
  2018-02-14  5:06 Can we make git clone --recurse-submodules --shallow-submodules work for commits that are not on tags or branches Ciro Santilli
@ 2018-02-14 19:18 ` Stefan Beller
  0 siblings, 0 replies; 2+ messages in thread
From: Stefan Beller @ 2018-02-14 19:18 UTC (permalink / raw)
  To: Ciro Santilli; +Cc: git

On Tue, Feb 13, 2018 at 9:06 PM, Ciro Santilli <ciro.santilli@gmail.com> wrote:
> I have a git repo with large submodules, notably the Linux kernel, and
> shallow clone saves a lot of time.
>
> However, QEMU is also a submodule, which I don't control, and QEMU has
> submodules which don't point to commits that are not on any tag or
> branch, so:
>
>  git clone --recurse-submodules --shallow-submodules git://git.qemu.org/qemu.git
>
> currently fails with:
>
> error: Server does not allow request for unadvertised object
> 22ead3e0bfdb87516656453336160e0a37b066bf
> Fetched in submodule path 'capstone', but it did not contain
> 22ead3e0bfdb87516656453336160e0a37b066bf. Direct fetching of that
> commit failed.
>
> on git 2.16.1.

In theory this should be easy. :)

In practice not so much, unfortunately. This is because cloning will just obtain
the latest tip of a branch (usually master). There is no mechanism in clone
to specify the exact sha1 that is wanted.

The wire protocol supports for asking exact sha1s, so that should be covered.
(Caveat: it only works if the server operator enables
uploadpack.allowReachableSHA1InWant which github has not AFAICT)

git-fetch allows to fetch arbitrary sha1, so as a workaround you can run a fetch
after the recursive clone by using "git submodule update" as that will use
fetches after the initial clone.


> Furthermore, I reproduce this locally with direct filesystem clones:
> https://github.com/cirosantilli/test-git-web-interface/blob/15335d3002a3e64fc5756b69fb832a733aa63fb9/shallow-submodule.sh#L158
> and on GitHub, so I'm guessing it is not just the settings for a
> specific server?
>
> Would it be possible to make that work, or are there fundamental
> reasons why it is not possible?

The wire protocol allows for it, so it should be possible fundamentally.
There is a redesign of the wire protocol going on currently, which
could allow for a new fetch mode that allows having just one
connection, which would only negotiate on the superproject and
infer the submodule parts from the superproject.


> Here is my use case repo, at the point of the ugly workaround I'm
> having to do: https://github.com/cirosantilli/linux-kernel-module-cheat/blob/a14c95346cfd9d2e7b2e475b0981aa71d819c20b/configure#L23
>
> Some more context:
> https://stackoverflow.com/questions/2144406/git-shallow-submodules/47374702#47374702
>
> This would make some embedded people happy.

Yes, a lot of submodule users would be happy if that
issue is solved.

Thanks for reporting,
Stefan

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-14  5:06 Can we make git clone --recurse-submodules --shallow-submodules work for commits that are not on tags or branches Ciro Santilli
2018-02-14 19:18 ` Stefan Beller

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox