Pushing tag from a partial clone

Pushing tag from a partial clone

[Son Luong Ngoc]

Hi folks,

At $day_job, we are trying to push tags to a repo from a partial clone copy.
However it seems like this push would requires the partial clone copy to download more objects?
Is this intended?

Reproduce:

	mkdir repo && cd repo
	git init
	git remote add origin git@domain.com:path/repo.git
	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
	git sparse-checkout init --cone
	git checkout --force <commit-id>
	git tag -a sluongng-test -m "Test push from partial clone"
	git push HEAD:refs/tags/sluongng-test
	<git starts to download objects>

Ideally we would like to be able to push tag from a shallow + partial clone repo without 
having to download extra objects if possible.
We would like to keep the required repo to the absolute minimum.
	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>

Creating and pushing tags should not require local repo to have trees/blobs in it?

Git version: 2.27.0

Cheers,
Son Luong.

Re: Pushing tag from a partial clone

[Derrick Stolee]

On 7/20/2020 7:44 AM, Son Luong Ngoc wrote:
> Hi folks,
> 
> At $day_job, we are trying to push tags to a repo from a partial clone copy.
> However it seems like this push would requires the partial clone copy to download more objects?
> Is this intended?
> 
> Reproduce:
> 
> 	mkdir repo && cd repo
> 	git init
> 	git remote add origin git@domain.com:path/repo.git
> 	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
> 	git sparse-checkout init --cone
> 	git checkout --force <commit-id>
> 	git tag -a sluongng-test -m "Test push from partial clone"
> 	git push HEAD:refs/tags/sluongng-test
> 	<git starts to download objects>
> 
> Ideally we would like to be able to push tag from a shallow + partial clone repo without 
> having to download extra objects if possible.
> We would like to keep the required repo to the absolute minimum.
> 	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>
> 
> Creating and pushing tags should not require local repo to have trees/blobs in it?
> 
> Git version: 2.27.0

Could you try this again with 2.28.0-rc1? I think Jonathan
Tan added the "no-fetch" flag in more places since 2.27.0,
and this might already be fixed.

Thanks,
-Stolee

Re: Pushing tag from a partial clone

[Son Luong Ngoc]

Hi,

> On Jul 20, 2020, at 14:18, Derrick Stolee <stolee@gmail.com> wrote:
> 
> On 7/20/2020 7:44 AM, Son Luong Ngoc wrote:
>> Hi folks,
>> 
>> At $day_job, we are trying to push tags to a repo from a partial clone copy.
>> However it seems like this push would requires the partial clone copy to download more objects?
>> Is this intended?
>> 
>> Reproduce:
>> 
>> 	mkdir repo && cd repo
>> 	git init
>> 	git remote add origin git@domain.com:path/repo.git
>> 	git fetch --filter=tree:0 --no-tags --prune origin <commit-id>
>> 	git sparse-checkout init --cone
>> 	git checkout --force <commit-id>
>> 	git tag -a sluongng-test -m "Test push from partial clone"
>> 	git push HEAD:refs/tags/sluongng-test
>> 	<git starts to download objects>
>> 
>> Ideally we would like to be able to push tag from a shallow + partial clone repo without 
>> having to download extra objects if possible.
>> We would like to keep the required repo to the absolute minimum.
>> 	git fetch --depth 1 --filter=tree:0 --no-tags --prune origin <commit-id>
>> 
>> Creating and pushing tags should not require local repo to have trees/blobs in it?
>> 
>> Git version: 2.27.0
> 
> Could you try this again with 2.28.0-rc1? I think Jonathan
> Tan added the "no-fetch" flag in more places since 2.27.0,
> and this might already be fixed.

I just freshly compiled from 'next' branch:

	> git version
	git version 2.28.0.rc1.139.gd6b33fda9d

And the problem still occurring:
	> mkdir scalar
	> cd scalar
	> git init
	Initialized empty Git repository in /Users/sluongngoc/work/booking/core/scalar/.git/
	# use my own fork here so that i have push permission
	> git remote add origin git@github.com:sluongng/scalar.git
	> git sparse-checkout init --cone
	> git fetch --filter=tree:0 --no-tags --prune origin 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
	remote: Enumerating objects: 2553, done.
	remote: Total 2553 (delta 0), reused 0 (delta 0), pack-reused 2553
	Receiving objects: 100% (2553/2553), 957.85 KiB | 1.06 MiB/s, done.
	Resolving deltas: 100% (74/74), done.
	From github.com:sluongng/scalar
	 * branch            4ba6c1c090e6e5a413e3ac2fc094205bd78f761e -> FETCH_HEAD
	> git tag -a test-tag -m 'test tag message' 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
	> git push origin refs/tags/test-tag:refs/tags/test-tag
	...<download start>

> 
> Thanks,
> -Stolee
> 

Thanks,
Son Luong.

Re: Pushing tag from a partial clone

[Jonathan Tan]

> I just freshly compiled from 'next' branch:
> 
> 	> git version
> 	git version 2.28.0.rc1.139.gd6b33fda9d
> 
> And the problem still occurring:
> 	> mkdir scalar
> 	> cd scalar
> 	> git init
> 	Initialized empty Git repository in /Users/sluongngoc/work/booking/core/scalar/.git/
> 	# use my own fork here so that i have push permission
> 	> git remote add origin git@github.com:sluongng/scalar.git
> 	> git sparse-checkout init --cone
> 	> git fetch --filter=tree:0 --no-tags --prune origin 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
> 	remote: Enumerating objects: 2553, done.
> 	remote: Total 2553 (delta 0), reused 0 (delta 0), pack-reused 2553
> 	Receiving objects: 100% (2553/2553), 957.85 KiB | 1.06 MiB/s, done.
> 	Resolving deltas: 100% (74/74), done.
> 	From github.com:sluongng/scalar
> 	 * branch            4ba6c1c090e6e5a413e3ac2fc094205bd78f761e -> FETCH_HEAD
> 	> git tag -a test-tag -m 'test tag message' 4ba6c1c090e6e5a413e3ac2fc094205bd78f761e
> 	> git push origin refs/tags/test-tag:refs/tags/test-tag
> 	...<download start>

Thanks for the reproduction steps. Is 4ba6c1c advertised as a ref by the
remote? If not, what is probably happening is that the client doesn't
realize that the server already has 4ba6c1c, so the client needs to
fetch 4ba6c1c's objects to send it to the server.

I am planning to see if I can add batch prefetching to pack-objects to
reduce the severity of similar situations (just one batch prefetch instead
of many one-by-one fetches), although that would work better with a blob
filter instead of a tree filter.