[Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

 Bare repository ORIG's master looks like this:

   o--o--o--o--v1--o--v2--o--o--o HEAD

 where v1 and v2 are (annotated) tagged commits.

 Repository SLAVE is a mirror clone of ORIG which
 (very deliberately!) lags behind (i.e., its HEAD
 is one of the earlier (and usually tagged) commits
 on ORIG).  SLAVE's master was like this:

   o--o--o--o--v1 HEAD

 We wanted to update its HEAD to v2, so did:

   git fetch ORIG tag v2

 This gave us:

   o--o--o--o--v1 HEAD
                 \ 
                  o--v2

 It did not update SLAVE's HEAD to v2, which we wanted.
 This was worked-around by editing refs/heads/master(?)
 but we don't want to do that again (esp. since we got
 it wrong the first time (Thank you back-ups!)).

 Whilst we want to switch to a push from ORIG to SLAVE
 model, until that happens (there are some IT issues),
 we are still fetching on SLAVE from ORIG.  Hence, how
 can we avoid the above issue; that is, what should we
 have done?  Searching various docs has failed to find
 any clew or answer.

 There are several GIT versions involved (all(?) are
 1.5-ish or later).  Below is a script to reproduce
 the situation (tested with v1.7.0.2).

cheers!
	-blf-

=====(cut here and below)===== demo.sh =====(git version 1.7.0.2)=====
#!/bin/bash

add_new_files() {
	touch   -- "$@"
	git add -- "$@"
	git commit -m "Added: $*"
}

set -xe

mkdir ORIG
cd ORIG
git init	# ORIG is bare in real case

add_new_files foo
add_new_files bar

git tag -a -m First v1

cd ..
git clone --bare --mirror ORIG SLAVE.git

cd ORIG

add_new_files xyzzy
add_new_files plover

git tag -a -m Update v2

add_new_files stuff
add_new_files more_stuff

cd ../SLAVE.git

git fetch --verbose origin tag v2

# SLAVE's master's HEAD has not changed ....  ;-(

git tag -l
git log --oneline master
git log --oneline v2
=====(cut here and above)===== demo.sh =====(git version 1.7.0.2)=====

Re: [Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Tomas Carnecky]

On 8/12/10 9:54 AM, Brian Foster wrote:
> 
>  Bare repository ORIG's master looks like this:
> 
>    o--o--o--o--v1--o--v2--o--o--o HEAD
> 
>  where v1 and v2 are (annotated) tagged commits.
> 
>  Repository SLAVE is a mirror clone of ORIG which
>  (very deliberately!) lags behind (i.e., its HEAD
>  is one of the earlier (and usually tagged) commits
>  on ORIG).  SLAVE's master was like this:
> 
>    o--o--o--o--v1 HEAD
> 
>  We wanted to update its HEAD to v2, so did:
> 
>    git fetch ORIG tag v2
> 
>  This gave us:
> 
>    o--o--o--o--v1 HEAD
>                  \ 
>                   o--v2
> 
>  It did not update SLAVE's HEAD to v2, which we wanted.
>  This was worked-around by editing refs/heads/master(?)
>  but we don't want to do that again (esp. since we got
>  it wrong the first time (Thank you back-ups!)).

Fetch only fetches commits. It doesn't update any local refs (other than
FETCH_HEAD). If you want to switch HEAD to that new tag, use checkout.

>  Whilst we want to switch to a push from ORIG to SLAVE
>  model, until that happens (there are some IT issues),
>  we are still fetching on SLAVE from ORIG.  Hence, how
>  can we avoid the above issue; that is, what should we
>  have done?  Searching various docs has failed to find
>  any clew or answer.

Is SLAVE a bare repo? If not, please see
https://git.wiki.kernel.org/index.php/GitFaq#non-bare.

tom

Re: [Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

On Thursday 12 August 2010 10:33:13 Tomas Carnecky wrote:

Tom,
 Thanks for the reply, but I suspect you didn't read
 the problem correctly ....

> On 8/12/10 9:54 AM, Brian Foster wrote:
> >  Bare repository ORIG's master looks like this:
> >[ ... ] 
> >  Repository SLAVE is a mirror clone of ORIG which
NOTE.......................^^^^^^
>[ ... ] 
> Fetch only fetches commits. It doesn't update any local refs (other than
> FETCH_HEAD). If you want to switch HEAD to that new tag, use checkout.

 No, a fetch in a mirror, when fetching all the way
 to the HEAD (e.g., a simple `git fetch origin'),
 does update the branch.  (You can easily modify the
 script I included to prove this.)

>[ ... ]
> Is SLAVE a bare repo?  [ ... ]

 Yes it is bare (because it's a mirror), as per the
 description, subject, and posted script.
cheers!
	-blf-

Re: [Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Tomas Carnecky]

On 8/12/10 12:38 PM, Brian Foster wrote:
> On Thursday 12 August 2010 10:33:13 Tomas Carnecky wrote:
> 
> Tom,
>  Thanks for the reply, but I suspect you didn't read
>  the problem correctly ....

I suspect I did but did not explain correctly (though I didn't realize
mirror implies bare, my bad).

> 
>> On 8/12/10 9:54 AM, Brian Foster wrote:
>>>  Bare repository ORIG's master looks like this:
>>> [ ... ] 
>>>  Repository SLAVE is a mirror clone of ORIG which
> NOTE.......................^^^^^^
>> [ ... ] 
>> Fetch only fetches commits. It doesn't update any local refs (other than
>> FETCH_HEAD). If you want to switch HEAD to that new tag, use checkout.

Oh boy, that explanation is really bad -.- What I meant is that if you
don't supply the <dst> part of the refspec it will only update
FETCH_HEAD. So 'git fetch origin master' will not update anything but
FETCH_HEAD.

>  No, a fetch in a mirror, when fetching all the way
>  to the HEAD (e.g., a simple `git fetch origin'),
>  does update the branch.  (You can easily modify the
>  script I included to prove this.)

There are different ways to invoke git fetch. Either way, fetch will
never modify HEAD. However, it can modify the branch that HEAD is
pointing to (in case HEAD is a symref). But you are not fetching a
branch, you are fetching a tag. So you need to use checkout after you
fetch it. Either 'git checkout FETCH_HEAD' or 'git checkout v2'.

tom

[SOLVED] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

On Thursday 12 August 2010 13:05:29 Tomas Carnecky wrote:
> On 8/12/10 12:38 PM, Brian Foster wrote:
> > On Thursday 12 August 2010 10:33:13 Tomas Carnecky wrote:
> >[ ... ]
> >> Fetch only fetches commits. It doesn't update any local refs (other than
> >> FETCH_HEAD). If you want to switch HEAD to that new tag, use checkout.
> 
> Oh boy, that explanation is really bad -.- What I meant is that if
> you don't supply the <dst> part of the refspec it will only update
> FETCH_HEAD. So 'git fetch origin master' will not update anything
> but FETCH_HEAD.

 I concur.  However, the “refspec” in this case is the obscure
 ‘tag TAGNAME’ (where the ‘tag’ is a keyword); see get-fetch(1)
 at the end of §OPTIONS where it describes “short-cut notations”.
 That does not seem to accept a :<dst>, nor would I be too sure
 what it meant if it did.

> >  No, a fetch in a mirror, when fetching all the way
> >  to the HEAD (e.g., a simple `git fetch origin'),
> >  does update the branch.  [ ... ]
> 
> There are different ways to invoke git fetch. Either way, fetch will
> never modify HEAD. However, it can modify the branch that HEAD is
> pointing to (in case HEAD is a symref). 

 Point taken, I was being sloppy.  We want the head of the
 relevant branch (master) to be updated in our bare (mirror)
 repository.

>                                         But you are not fetching a
> branch, you are fetching a tag. So you need to use checkout after you
> fetch it. Either 'git checkout FETCH_HEAD' or 'git checkout v2'.

 NO.  ‘get checkout ...’ does not work in a bare repository.
 (You can modify my trivial posted script to prove this.)

 However, your comment got me to thinking.  This _does_ work:

    git reset --soft FETCH_HEAD

 (And, I presume without testing, so would ‘... v2’ ?)
 Given that git-reset(1)'s purpose is to change the branch's
 head (or to change HEAD as I've been incorrectly describing
 it), that does make sense.

cheers!
	-blf-

Re: [SOLVED] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Tomas Carnecky]

On 8/12/10 2:16 PM, Brian Foster wrote:
>  NO.  ‘get checkout ...’ does not work in a bare repository.
>  (You can modify my trivial posted script to prove this.)
> 
>  However, your comment got me to thinking.  This _does_ work:
> 
>     git reset --soft FETCH_HEAD
> 
>  (And, I presume without testing, so would ‘... v2’ ?)
>  Given that git-reset(1)'s purpose is to change the branch's
>  head (or to change HEAD as I've been incorrectly describing
>  it), that does make sense.

I expected git checkout to work even in a bare repo - as you can see I
don't work that often inside bare repos :).

You should use update-ref instead of reset. The problem with reset is
that it changes the branch and not HEAD. Say initially your mirror had
HEAD as symref pointing to refs/heads/master. When you run reset --soft
v2 your master branch will now point to the same commit as the tag v2
(and HEAD will be still pointing to refs/heads/master). When you use
'git update-ref --no-deref HEAD v2' it will modify HEAD directly and
have it point directly to the same commit as v2 (also called detached HEAD).

tom

Re: [SOLVED] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

On Thursday 12 August 2010 14:32:41 Tomas Carnecky wrote:
> On 8/12/10 2:16 PM, Brian Foster wrote:
> >[ ... ]  This _does_ work:
> > 
> >     git reset --soft FETCH_HEAD
> >[ ... ] 
> >  Given that git-reset(1)'s purpose is to change the branch's
> >  head (or to change HEAD as I've been incorrectly describing
> >  it), that does make sense.
> 
> I expected git checkout to work even in a bare repo - as you can
> see I don't work that often inside bare repos :).

 <<<giggles>>>  No problem!  (I don't either, this situation was
 unexpected, but as SLAVE lagging behind ORIG is anticipated as
 being the usual case, we need to understand it.)

> You should use update-ref instead of reset. The problem with reset is
> that it changes the branch and not HEAD. Say initially your mirror had
> HEAD as symref pointing to refs/heads/master.

 Correct, it does.

>                                               When you run reset --soft
> v2 your master branch will now point to the same commit as the tag v2
> (and HEAD will be still pointing to refs/heads/master).

 Correct, that is what happens.  It also looks Ok to me,
 nor have any tests hit any snags.  It also happens to
 match how we (eventually) manually repaired SLAVE.

>                                                         When you use
> 'git update-ref --no-deref HEAD v2' it will modify HEAD

 NO, it gets an error (following is from a modified version
 of my previously-posted script):
────────────────────────────────────────────────────────
 ...
+ cd ../SLAVE.git                                           
+ git fetch --verbose origin tag v2                         
 ...
From /work/tmp/git/ORIG                      
 * [new tag]         v2         -> v2        
+ git update-ref --no-deref HEAD v2          
error: Trying to write non-commit object 2bc324e6a68cb3704448c9f63ddc3bc0260c0b48 to branch HEAD                                                                                
fatal: Cannot update the ref 'HEAD'.                                                    
$ 
────────────────────────────────────────────────────────

 Using ‘git update-ref --no-deref HEAD FETCH_HEAD’ does
 not error, and, as you say (below), does seem to give
 me a detached head.

> 'git update-ref --no-deref HEAD v2' it will modify HEAD directly and
> have it point directly to the same commit as v2 (also called detached HEAD).

 I do_not_ want a detached head!  This has perhaps been
 confused by my improperly saying “update HEAD” when I
 meant “update the master branch”.  If you look back at
 my originally-posted diagrams, it's a reasonable guess
 I don't want a detached head.  Having HEAD continue to
 be a symref to refs/heads/master (which, in this case,
 is the same commit as v2) is correct.

 Thanks for your help, advice, and hints.
cheers!
	-blf-

Re: [SOLVED] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Tomas Carnecky]

On 8/12/10 3:22 PM, Brian Foster wrote:
> On Thursday 12 August 2010 14:32:41 Tomas Carnecky wrote:
>>                                                         When you use
>> 'git update-ref --no-deref HEAD v2' it will modify HEAD
> 
>  NO, it gets an error (following is from a modified version
>  of my previously-posted script):
> + git update-ref --no-deref HEAD v2          
> error: Trying to write non-commit object 2bc324e6a68cb3704448c9f63ddc3bc0260c0b48 to branch HEAD                                                                                

Oh the subtleties when working with plumbing:
git update-ref --no-deref HEAD v2^{commit}

>  Using ‘git update-ref --no-deref HEAD FETCH_HEAD’ does
>  not error, and, as you say (below), does seem to give
>  me a detached head.
> 
>> 'git update-ref --no-deref HEAD v2' it will modify HEAD directly and
>> have it point directly to the same commit as v2 (also called detached HEAD).
> 
>  I do_not_ want a detached head!  This has perhaps been
>  confused by my improperly saying “update HEAD” when I
>  meant “update the master branch”.  If you look back at
>  my originally-posted diagrams, it's a reasonable guess
>  I don't want a detached head.  Having HEAD continue to
>  be a symref to refs/heads/master (which, in this case,
>  is the same commit as v2) is correct.

Just make sure your tools and scripts don't break when you reset the
branch like that (especially when the update is non-fast forward as
fetch+reset won't warn you when this happens).

tom

Re: [Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brandon Casey]

Brian Foster wrote:
>  Bare repository ORIG's master looks like this:
> 
>    o--o--o--o--v1--o--v2--o--o--o HEAD
> 
>  where v1 and v2 are (annotated) tagged commits.
> 
>  Repository SLAVE is a mirror clone of ORIG which
>  (very deliberately!) lags behind (i.e., its HEAD
>  is one of the earlier (and usually tagged) commits
>  on ORIG).  SLAVE's master was like this:
> 
>    o--o--o--o--v1 HEAD
> 
>  We wanted to update its HEAD to v2, so did:
> 
>    git fetch ORIG tag v2
> 
>  This gave us:
> 
>    o--o--o--o--v1 HEAD
>                  \ 
>                   o--v2
> 
>  It did not update SLAVE's HEAD to v2, which we wanted.

Are you using --mirror only so that the branch pointer in
the mirror repository will be updated when you fetch?

If you are not really interested in having a "real" mirror,
then maybe you should set your mirror up to track a
specific branch (or branches) in the mirrored repository.

You could have a branch named for example "for_mirror/master",
in the mirrored repository (ORIG) that you would prepare.
You could update the for_mirror/master branch when you were
ready using 'git branch' like this:

   git branch -f for_mirror/master v2

In the mirror repository (SLAVE), you would update the
fetchspec so that the mirror mirrored the branches below
the "for_mirror" namespace in the remote repository
like this:

   fetch = +refs/heads/for_mirror/*:refs/heads/*

Then, a simple 'git fetch' would fetch the updates (including
tags) and update the branch pointer in the mirror like you want
it to do.  Tracking multiple branches this way is possible just
by creating another branch in the ORIG repository with the proper
name.

-Brandon

Re: [SOLVED] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

On Thursday 12 August 2010 15:34:24 Tomas Carnecky wrote:
> On 8/12/10 3:22 PM, Brian Foster wrote:
> > On Thursday 12 August 2010 14:32:41 Tomas Carnecky wrote:
> >>                                                         When you use
> >> 'git update-ref --no-deref HEAD v2' it will modify HEAD
> > 
> >  NO, it gets an error [ ... ]
> 
> Oh the subtleties when working with plumbing:
>    git update-ref --no-deref HEAD v2^{commit}

 Yes, that updates HEAD, leaving SLAVE in the odd state
 of HEAD being v2 and master's head being v1 (which I
 presume is another example of detached head?).

> >[ ... ]
> >  I do_not_ want a detached head!  This has perhaps been
> >  confused by my improperly saying “update HEAD” when I
> >  meant “update the master branch”.  [ ... ]
> >                                 Having HEAD continue to
> >  be a symref to refs/heads/master (which, in this case,
> >  is the same commit as v2) is correct.
> 
> Just make sure your tools and scripts don't break when you reset the
> branch like that (especially when the update is non-fast forward as
> fetch+reset won't warn you when this happens).

 Understood.  And I'm not too worried about that happening,
 albeit the current procedures are perhaps not as robust as
 they could or should be.  Again, thanks for the suggestions
 and help.

cheers!
	-blf-

Re: [Q] `git fetch tag NAME' into mirror repo does not update HEAD, what to do?

[Brian Foster]

On Thursday 12 August 2010 19:02:49 Brandon Casey wrote:
> Brian Foster wrote:
> >  Bare repository ORIG's master looks like this [ I've updated
> >  all of the diagrams slightly to remove some confusion  -blf]:
> > 
> >    o--o--o--o--v1--o--v2--o--o--o [master &] HEAD
> > 
> >  where v1 and v2 are (annotated) tagged commits.
> > 
> >  Repository SLAVE is a mirror clone of ORIG which
> >  (very deliberately!) lags behind (i.e., [SLAVE's master's head]
> >  is one of the earlier (and usually tagged) commits
> >  on ORIG).  SLAVE's master was like this:
> > 
> >    o--o--o--o--v1 [master &] HEAD
> > 
> >  We wanted to update [SLAVE's master's head] to v2, so did:
> > 
> >    git fetch ORIG tag v2
> > 
> >  This gave us:
> > 
> >    o--o--o--o--v1 [master &] HEAD
> >                  \ 
> >                   o--v2
> > 
> >  It did not update SLAVE's [master's head] to v2, which we wanted[:]

       o--o--o--o--v1--o--v2 [master &] HEAD

> Are you using --mirror only so that the branch pointer in
> the mirror repository will be updated when you fetch?

 Correct.

> If you are not really interested in having a "real" mirror,
> then maybe you should set your mirror up to track a
> specific branch (or branches) in the mirrored repository.

 Now that seems like an excellent idea!

 The embarrassing thing is, we actually do _do_ something
 similar elsewhere in the process.  So we haven't got any
 great excuse for not thinking of it ....  ;-\ 

> You could have a branch named for example "for_mirror/master",
> in the mirrored repository (ORIG) that you would prepare.
> You could update the for_mirror/master branch when you were
> ready using 'git branch' like this:
> 
>    git branch -f for_mirror/master v2
> 
> In the mirror repository (SLAVE), you would update the
> fetchspec so that the mirror mirrored the branches below
> the "for_mirror" namespace in the remote repository
> like this:
> 
>    fetch = +refs/heads/for_mirror/*:refs/heads/*
> 
> Then, a simple 'git fetch' would fetch the updates (including
> tags) and update the branch pointer in the mirror like you want
> it to do.  Tracking multiple branches this way is possible just
> by creating another branch in the ORIG repository with the proper
> name.

 Another nice point about the above is it could deal
 with another procedural issue we have, which is the
 lack of a “staging area” for final test (a “pre-SLAVE”).
 Whilst we haven't (yet) had any (known) screw-ups in
 the final released system (what “mirror” SLAVE contains),
 it can happen.  The known screw-ups were all detected
 before it went to SLAVE, but by accident (IM(H?)O).

 Thanks for the idea/observation.

cheers!
	-blf-