Behavior of 'git fetch' for commit hashes

Behavior of 'git fetch' for commit hashes

[eero.aaltonen]

Hello,

I am seeing greatly different behavior for 'git fetch' commands on
version 2.7.4 and 2.13.1 when the argument to fetch is a commit hash.

I am working on a custom Continuous Integration (CI) system. The
projects have dependencies that are simply fetched as repositories to a
project subdirectory.

In order to prevent a race condition in updating branches, I want to
checkout the actual commit that was built in a previous job.

Environment
===========
Problem occurs with Git clients
- 2.7.4
- 2.13.1

and Git server
- Atlassian Bitbucket v4.14.3
- git version 2.12.0


The problem can be reproduced as follows, with
- <authoritativeUrl> = URL to primary repository
- <forkUrl> = URL to fork of the primary
- <sha1> commit hash in a recent branch

1. git clone <authoritativeUrl>
2. cd resulting_directory
3. git fetch <forkUrl> <sha1>

For version 2.7.4
=================
Git exits with exit code 1.

However, if I first do 'git fetch <branch>', then 'git fetch <sha1> will
also work

 * branch   <sha1> -> FETCH_HEAD

For version 2.13.3
==================
Git exits with exit code 128 and message
fatal: Couldn't find remote ref

However, the workaround for descbibed abot for git version 2.7.4 no
longer works. The result is always
fatal: Couldn't find remote ref

Desired result
==============
Commit is in .git/FETCH_HEAD and can be checked out.


I want to checkout a specific commit without creating any extra named
remotes in the local git clone.

Finally,
What is the expected behavior for 'git fetch' in this case?
Is there some other way I can achieve my goals?

-- 
Best regards,
Eero Aaltonen

Re: Behavior of 'git fetch' for commit hashes

[Stefan Beller]

+cc Jonathan who did some work in the area.

On Mon, Jun 19, 2017 at 5:09 AM,  <eero.aaltonen@vaisala.com> wrote:
> Hello,
>
> I am seeing greatly different behavior for 'git fetch' commands on
> version 2.7.4 and 2.13.1 when the argument to fetch is a commit hash.
>
> I am working on a custom Continuous Integration (CI) system. The
> projects have dependencies that are simply fetched as repositories to a
> project subdirectory.
>
> In order to prevent a race condition in updating branches, I want to
> checkout the actual commit that was built in a previous job.
>
> Environment
> ===========
> Problem occurs with Git clients
> - 2.7.4
> - 2.13.1
>
> and Git server
> - Atlassian Bitbucket v4.14.3
> - git version 2.12.0
>
>
> The problem can be reproduced as follows, with
> - <authoritativeUrl> = URL to primary repository
> - <forkUrl> = URL to fork of the primary
> - <sha1> commit hash in a recent branch
>
> 1. git clone <authoritativeUrl>
> 2. cd resulting_directory
> 3. git fetch <forkUrl> <sha1>
>
> For version 2.7.4
> =================
> Git exits with exit code 1.
>
> However, if I first do 'git fetch <branch>', then 'git fetch <sha1> will
> also work
>
>  * branch   <sha1> -> FETCH_HEAD
>
> For version 2.13.3
> ==================
> Git exits with exit code 128 and message
> fatal: Couldn't find remote ref
>
> However, the workaround for descbibed abot for git version 2.7.4 no
> longer works. The result is always
> fatal: Couldn't find remote ref
>
> Desired result
> ==============
> Commit is in .git/FETCH_HEAD and can be checked out.
>
>
> I want to checkout a specific commit without creating any extra named
> remotes in the local git clone.
>
> Finally,
> What is the expected behavior for 'git fetch' in this case?
> Is there some other way I can achieve my goals?
>
> --
> Best regards,
> Eero Aaltonen

Re: Behavior of 'git fetch' for commit hashes

[Jonathan Tan]

On Mon, 19 Jun 2017 12:09:28 +0000
<eero.aaltonen@vaisala.com> wrote:

> For version 2.7.4
> =================
> Git exits with exit code 1.
> 
> However, if I first do 'git fetch <branch>', then 'git fetch <sha1> will
> also work
> 
>  * branch   <sha1> -> FETCH_HEAD

I suspect that what is happening is that 'git fetch <branch>' also
downloads the commit referenced by <sha1>, so the subsequent 'git fetch
<sha1>' is a no-op (except setting FETCH_HEAD).

> For version 2.13.3
> ==================
> Git exits with exit code 128 and message
> fatal: Couldn't find remote ref
> 
> However, the workaround for descbibed abot for git version 2.7.4 no
> longer works. The result is always
> fatal: Couldn't find remote ref

I'll take a look into this.

> Desired result
> ==============
> Commit is in .git/FETCH_HEAD and can be checked out.
> 
> 
> I want to checkout a specific commit without creating any extra named
> remotes in the local git clone.
> 
> Finally,
> What is the expected behavior for 'git fetch' in this case?
> Is there some other way I can achieve my goals?

I'll take a look into the expected behavior, but if my assumptions are
correct, you should be able to just checkout the commit you want after
fetching the branch:

  git fetch <forkUrl> <branch>
  git checkout <sha1>

Re: Behavior of 'git fetch' for commit hashes

[Jonathan Tan]

On Mon, 19 Jun 2017 10:49:36 -0700
Jonathan Tan <jonathantanmy@google.com> wrote:

> On Mon, 19 Jun 2017 12:09:28 +0000
> <eero.aaltonen@vaisala.com> wrote:
> 
> > For version 2.13.3

Firstly, exactly which version of Git doesn't work? I'm assuming 2.13.1
(as written elsewhere in your e-mail), since 2.13.3 doesn't exist.

> > However, the workaround for descbibed abot for git version 2.7.4 no
> > longer works. The result is always
> > fatal: Couldn't find remote ref
> 
> I'll take a look into this.

I tried to reproduce with this script, but it seems to pass even at
2.13.1:

    #!/bin/bash
    rm -rf ~/tmp/x &&
    make --quiet &&
    ./git init ~/tmp/x &&
    ./git -C ~/tmp/x fetch --quiet ~/gitpristine/git master:foo &&
    ./git -C ~/tmp/x fetch ~/gitpristine/git "$(git -C ~/gitpristine/git rev-parse master^)"
    exit $?

Commenting out the first fetch line produces, as expected:

    error: Server does not allow request for unadvertised object <hash>

And I have not seen the "fatal: Couldn't find remote ref" error you
describe.

As Stefan alluded to in his earlier e-mail [1], I made a change recently
to how "git fetch" handles literal SHA-1s in commit fdb69d3
("fetch-pack: always allow fetching of literal SHA1s", 2017-05-15). In
this case, I don't think that's the cause of this issue (since that
change, if anything, makes things more permissive, not less) but that
might be a point of interest in an investigation.

[1] https://public-inbox.org/git/CAGZ79kbFeptKOfpaZ23yK=Zw9mJ0_evqPstHKkD1HSCaP_pC5g@mail.gmail.com/

Re: Behavior of 'git fetch' for commit hashes

[eero.aaltonen]

On 20.06.2017 01:26, Jonathan Tan wrote:
> On Mon, 19 Jun 2017 10:49:36 -0700
> Jonathan Tan <jonathantanmy@google.com> wrote:
> 
>> On Mon, 19 Jun 2017 12:09:28 +0000
>> <eero.aaltonen@vaisala.com> wrote:
>>
>>> For version 2.13.3
> 
> Firstly, exactly which version of Git doesn't work? I'm assuming 2.13.1
> (as written elsewhere in your e-mail), since 2.13.3 doesn't exist.

Yes. 2.13.1. I should stick to copy-pasting.

> I tried to reproduce with this script, but it seems to pass even at
> 2.13.1:
> 
>     #!/bin/bash
>     rm -rf ~/tmp/x &&
>     make --quiet &&
>     ./git init ~/tmp/x &&
>     ./git -C ~/tmp/x fetch --quiet ~/gitpristine/git master:foo &&
>     ./git -C ~/tmp/x fetch ~/gitpristine/git "$(git -C ~/gitpristine/git rev-parse master^)"
>     exit $?
> 
> Commenting out the first fetch line produces, as expected:
> 
>     error: Server does not allow request for unadvertised object <hash>
> 
> And I have not seen the "fatal: Couldn't find remote ref" error you
> describe.

I am now getting the same "unadvertised object" error. The "remote ref"
error it seems was due to missing the last character of the SHA1 :/

Now with that resolved, the "fetch branch; fetch commit" also works just
as before.

> I'll take a look into the expected behavior, but if my assumptions are
> correct, you should be able to just checkout the commit you want after
> fetching the branch:
> 
>   git fetch <forkUrl> <branch>
>   git checkout <sha1>

This actually produces
fatal: reference is not a tree: <sha1>

But I can use the above dual fetch. Seems this problem was more between
the keyboard and chair. I am however glad to hear that literal SHA1s are
becoming fetchable.

Thanks for investigating,
-- 
Eero Aaltonen