Fetch origin/HEAD during git fetch

Fetch origin/HEAD during git fetch

[Dominik Salvet]

Hello Git community!
Imagine a repository initialised using git init. Then, a remote (e.g.,
origin) is added. Now, I want to fetch all branches, tags and commits
from the prepared remote to fill the local repository up for the first
time and do it as quick as possible. I want the origin/HEAD reference
to be fetched as well, and this is where the problem is - I don't know
how to do it optimally.

To summarise it, I want the following:

* Prepare a repository as quick as possible
* No local branches in the repository
* No files in working tree after downloading
* Working tree must be available (for further checkouts)
* Keep the information about its default branch (origin/HEAD)
* Updates can be fetched using a git fetch (ideally a plain one)
* The repository will not be ever used for pushing (it does not have
to be disabled)

Three sub-optimal solutions I have considered:

1) `git fetch origin && git remote set-head origin -a`
2) `git fetch origin +refs/heads/*:refs/remotes/origin/*
HEAD:refs/remotes/origin/HEAD`
3) instead of git init and remote, use directly `git clone --no-checkout`

The first solution is not suitable due its delay caused by remote
access (2 separate invocations). For smaller repositories, delays of
these individual commands are almost comparable. However, this
solution is semantically exactly what I want.

The problem with the second solution is that it will create
origin/HEAD as a commit hash, not as a symbolic reference to a branch
(e.g., master). When master branch gets updated, it may cause problems
with further update fetching using a plain git fetch. Also, this
solution doesn't look very familiar and stable.

The third solution has several problems. The first one is the created
default local branch. So delete it. However, to delete it, I need to
checkout another commit and that modifies working tree, which can
cause unnecessary delay (namely for bigger repositories) as it may
(and will in most cases) be checked out to another commit later. Then
I can finally remove the local branch (after getting its name,
though).

If you wonder which applications I need such restrictions for - I am a
developer of GitPack, a very simple Git-based package manager, and I
would like to solve the mentioned problem to improve storing
downloaded repositories in GitPack cache. It would allow GitPack
projects to use a different default branch than master branch, which
is currently hard-coded in GitPack.

I hope everything is clear. If not, do not hesitate and ask me. I
would really appreciate your help.

Thank you for all your effort
-- 
Dominik Salvet

Re: Fetch origin/HEAD during git fetch

[Jonathan Nieder]

Hi!

Dominik Salvet wrote:

> 1) `git fetch origin && git remote set-head origin -a`
> 2) `git fetch origin +refs/heads/*:refs/remotes/origin/*
> HEAD:refs/remotes/origin/HEAD`
> 3) instead of git init and remote, use directly `git clone --no-checkout`
>
> The first solution is not suitable due its delay caused by remote
> access (2 separate invocations). For smaller repositories, delays of
> these individual commands are almost comparable. However, this
> solution is semantically exactly what I want.

Interesting.

For the specific case, it sounds like some kind of "git fetch
--also-set-head" would do the trick for you.  As you can see, I'm awful
at naming command line options --- aside from the need to give it a
better name, any others thoughts on that?  Would you be interested in
taking a stab at implementing it?

For the more general case, there's been some discussion of fetching
and pushing symrefs on-list before.  [1] discusses one possible UI.

[...]
> The third solution has several problems. The first one is the created
> default local branch. So delete it.

Hm, I don't follow.  Does "git checkout --orphan" do what you're
looking for?

Thanks and hope that helps,
Jonathan

[1] https://public-inbox.org/git/20180814214723.GA667@sigill.intra.peff.net/

Re: Fetch origin/HEAD during git fetch

[Dominik Salvet]

Hi Jonathan,
Thank you for your time. I would be for something like `git fetch
--head` or `git fetch --remote-head` as the name. It also matches `git
fetch --tags` - fetch tags from remote, in addition to whatever else.
Aside from that, there are no further comments from my side. I would
be really happy to see the option in newer versions of Git (though, I
would be more happier being able to implement it in current ones...).
However, I won't be able to implement it myself for two reasons; a)
not enough skill in C and b) unfortunately, I do not have enough time.
Nevertheless, I can assist during the development of this feature if
approved and if there is someone who actually writes the code.

Another solution, on the other hand, could be the one from the
provided link [1] - use ~ for symrefs. That would also solve my
problem and since it would be equivalent to `git remote set-head
origin -a`, I am perfectly fine with that. Just run `git config --add
remote.origin.fetch ~HEAD:refs/remotes/origin/HEAD` after setting up
the origin remote and then just use `git fetch`.

About the `git checkout --orphan`: it also modifies the work tree in
my case (files in local repository), which I don't want to. Still, if
I want to create a branch to delete another branch (default clone
local branch), that itself is counterproductive - I don't want any
local branches. There is another problem, though. What name to choose
for `git checkout --orphan <name>` not to conflict with a default
branch (that could be anything) of the repository? Yep, there is a
solution for that but I don't want to make things over-complicated. I
would rather not use the `git clone` solution as long as there is no
`git clone --no-checkout-and-no-local-branches`-like option/s.

However, it looks like I have found an optimal workaround for my case.
Use git init, setup git remote and also run `git config --add
remote.origin.fetch HEAD:refs/remotes/origin/HEAD`. Now, using a plain
`git fetch` for fetching updates also fetches the current default
remote commit in origin/HEAD each time used, so no symbolic reference
to a branch might not be required as origin/HEAD points to the latest
commit of the default branch in both cases. Symbolic reference may be
beneficient in case when not fetching origin/HEAD while fetching the
branch pointed by origin/HEAD reference, which may fail in case the
remote updates its origin/HEAD anyways (e.g., on GitHub - repository
Settings -> Branches -> Default branch).

Due to my workaround, I guess now it's up to you, developers, whether
developing a `git fetch --head` option or ~ for symrefs worth it for
you. Anyway, if you find any pitfalls in my workaround, please let me
know. If you are rather interested in the actual implementation, take
a look at GitPack hosted on github.com/dominiksalvet/gitpack, the
affected file is src/gitpack (search for 'git config').

Have a nice day,

On Wed, 9 Oct 2019 at 04:25, Jonathan Nieder <jrnieder@gmail.com> wrote:
>
> Hi!
>
> Dominik Salvet wrote:
>
> > 1) `git fetch origin && git remote set-head origin -a`
> > 2) `git fetch origin +refs/heads/*:refs/remotes/origin/*
> > HEAD:refs/remotes/origin/HEAD`
> > 3) instead of git init and remote, use directly `git clone --no-checkout`
> >
> > The first solution is not suitable due its delay caused by remote
> > access (2 separate invocations). For smaller repositories, delays of
> > these individual commands are almost comparable. However, this
> > solution is semantically exactly what I want.
>
> Interesting.
>
> For the specific case, it sounds like some kind of "git fetch
> --also-set-head" would do the trick for you.  As you can see, I'm awful
> at naming command line options --- aside from the need to give it a
> better name, any others thoughts on that?  Would you be interested in
> taking a stab at implementing it?
>
> For the more general case, there's been some discussion of fetching
> and pushing symrefs on-list before.  [1] discusses one possible UI.
>
> [...]
> > The third solution has several problems. The first one is the created
> > default local branch. So delete it.
>
> Hm, I don't follow.  Does "git checkout --orphan" do what you're
> looking for?
>
> Thanks and hope that helps,
> Jonathan
>
> [1] https://public-inbox.org/git/20180814214723.GA667@sigill.intra.peff.net/



--
Dominik Salvet