Optionally include submodules in git archive - status?

Optionally include submodules in git archive - status?

[Axel Obermeier]

Dear git developers & community,

I'm trying to find out the status of including submodules in git archive
- I'm sure this has come up before, because it creates lots of pain,
especially in packaging [0].

Github in particular punts on including submodules in the automatically
generated tarballs (which are therefore often broken, i.e. not including
all necessary parts to build), and this is in part because the underlying
git archive command does not provide that option.

I could not find a bug tracker for git (though I noted with interest
from [1] that the git-scm website has one...), and could not easily find
something on the ML [2].

It's creating enough friction for me that I feel tempted to implement it
(e.g. with a switch to recurse one or all levels of submodules), even
though I don't know the git codebase at all. In any case, before rushing
ahead, I wanted to check first what the status of this topic is.

Best regards
H.

[0] https://github.com/dear-github/dear-github/issues/214
[1] https://git-scm.com/community
[2] https://public-inbox.org/git/?q=submodule+%22git+archive%22

Re: Optionally include submodules in git archive - status?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2142 bytes --]

On 2021-06-11 at 07:21:29, Axel Obermeier wrote:
> Dear git developers & community,
> 
> I'm trying to find out the status of including submodules in git archive
> - I'm sure this has come up before, because it creates lots of pain,
> especially in packaging [0].

Nobody has implemented it yet.  It could theoretically be implemented
for submodules where the submodule is already downloaded and populated
on disk, but if the submodule isn't populated, then the data won't exist
and it wouldn't be possible to put it in the archive.

So it can be done, but it's a bit tricky.

> Github in particular punts on including submodules in the automatically
> generated tarballs (which are therefore often broken, i.e. not including
> all necessary parts to build), and this is in part because the underlying
> git archive command does not provide that option.

It is unlikely that GitHub will implement this anytime soon.  GitHub
essentially invokes git archive to generate its archives, and even when
git archive supports that, the submodules will usually live on a
different set of file servers than the main repository, so the data will
likely not be accessible.

For Git LFS, we issue our own source tarballs for different reasons (the
fact that git archive is not guaranteed to produce stable archives and
we want to provide signed hashes), so at least for now, the solution
would be to provide your own tarballs if you need something special.

> I could not find a bug tracker for git (though I noted with interest
> from [1] that the git-scm website has one...), and could not easily find
> something on the ML [2].

The list is the place to report bugs and request new features, so you're
in the right place.

> It's creating enough friction for me that I feel tempted to implement it
> (e.g. with a switch to recurse one or all levels of submodules), even
> though I don't know the git codebase at all. In any case, before rushing
> ahead, I wanted to check first what the status of this topic is.

We would love to see a patch.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]