git clone --filter=blob:limit=X interaction with submodules?

git clone --filter=blob:limit=X interaction with submodules?

[Luke Diamand]

I noticed that if I do git clone with submodules, using
"--filter=blob:limit=whatever" that the filter is applied in the super
project, but appears to be ignored in the sub modules.

Is that expected behaviour? Or am I just doing it wrong?

I'm using git 2.27.0.90.geebb51ba8c.

Thanks
Luke

Re: git clone --filter=blob:limit=X interaction with submodules?

[Jonathan Tan]

> I noticed that if I do git clone with submodules, using
> "--filter=blob:limit=whatever" that the filter is applied in the super
> project, but appears to be ignored in the sub modules.
> 
> Is that expected behaviour? Or am I just doing it wrong?
> 
> I'm using git 2.27.0.90.geebb51ba8c.
> 
> Thanks
> Luke

Yes, that's expected. The main technical reason is that a lot of the
subsequent submodule operations may read objects from the submodules,
and those could trigger on-demand fetches (if the submodules were
themselves partial clones) - but right now the fetch code is designed to
run only on one repository.

For those who would like to improve this state of affairs, one step in
the right direction would be to update the fetch and transport code to
support an arbitrary "struct repository" passed in, and not just operate
on the_repository.

Re: git clone --filter=blob:limit=X interaction with submodules?

[Luke Diamand]

On Thu, 18 Jun 2020 at 00:20, Jonathan Tan <jonathantanmy@google.com> wrote:
>
> > I noticed that if I do git clone with submodules, using
> > "--filter=blob:limit=whatever" that the filter is applied in the super
> > project, but appears to be ignored in the sub modules.
> >
> > Is that expected behaviour? Or am I just doing it wrong?
> >
> > I'm using git 2.27.0.90.geebb51ba8c.
> >
> > Thanks
> > Luke
>
> Yes, that's expected. The main technical reason is that a lot of the
> subsequent submodule operations may read objects from the submodules,
> and those could trigger on-demand fetches (if the submodules were
> themselves partial clones) - but right now the fetch code is designed to
> run only on one repository.

Would this also explain why I get errors when syncing my superproject:

$ git pull --rebase
remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 844 bytes | 844.00 KiB/s, done.
Resolving deltas: 100% (2/2), completed with 2 local objects.
From ssh://whatever/somerepo
   7a9c9e1eeee..3325285b122  master     -> origin/master
fatal: git upload-pack: not our ref edcedb543a8b85c659ab7cb507bdb0211fb21bdf
fatal: remote error: upload-pack: not our ref
edcedb543a8b85c659ab7cb507bdb0211fb21bdf
The SSH command failed, please check the server logs for details.

This happens consistently when pulling from a repo that was originally
cloned using --filter. The same repo without the filtering does not
have this problem.

>
> For those who would like to improve this state of affairs, one step in
> the right direction would be to update the fetch and transport code to
> support an arbitrary "struct repository" passed in, and not just operate
> on the_repository.