cache status after git pull

cache status after git pull

[tony.luck]

>* When the branch head pointed by $GIT_DIR/HEAD changes while
> the index file and working tree are looking the other way
> (e.g. somebody pushed into your repository, or you ran "git
> fetch" to update the ref your working tree is on), "git
> checkout" without -f gets confused.  Figure out a good way to
> handle this.

Aha ... is this the problem that caught me out last week (when
I ended up with 10 extra files attached to one of my commits)? At
the time the blame was placed on a failed merge not being backed
out correctly.  But I only had the failed merge because "get checkout"
had failed to switch branches (and not provided an exit code to
stop my script from trying the merge).

Here's what I did this morning.

1) Updated my "linus" branch:

  $ git checkout linus && git pull linus

This appeared to work just fine ... except that when I
check the status of my tree I see:

  $ git status
  #
  # Updated but not checked in:
  #   (will commit)
  #
  #       modified: arch/ia64/pci/pci.c
  #       modified: arch/ppc64/kernel/setup.c
  #       modified: arch/sparc64/kernel/pci.c
  #       modified: arch/x86_64/defconfig
  #       modified: drivers/block/cfq-iosched.c
  #       modified: include/asm-m68k/page.h
  #       modified: kernel/cpuset.c
  #
  #
  # On branch refs/heads/linus
  
Which looks like a set of landmines just waiting for me to
step on them!

Today these didn't bite me.  "git checkout release" worked
and switched to my release branch (and git status went back
to saying "nothing to commit").  But in the past I think
this is the situation that has caused "git checkout" to fail
with the "fatal: Entry 'blah' would be overwritten by merge. Cannot merge."

-Tony

Re: cache status after git pull

[Junio C Hamano]

tony.luck@intel.com writes:

> Aha ... is this the problem that caught me out last week (when
> I ended up with 10 extra files attached to one of my commits)?

Plausible.

> 1) Updated my "linus" branch:
>
>   $ git checkout linus && git pull linus

I would assume that just after "git checkout linus" before "git
pull linus", running "git diff -r linus" would have said
nothing.

The second command, "git pull linus", would internally run "git
fetch linus".  It depends on how your shorthand "linus" is
defined, but if it is set to update (either overwrite or
fast-forward) the "linus" branch head, then your HEAD pointer
would be updated without updating the index and working tree.
This is bad because now you are telling git that your working
tree is based on updated "linus" branch head, and what you
_could_ commit on top of it is the same thing as what old
"linus" branch head commit used to have.  That's why "git
status" output shows the minefield.

If I keep copies of foreign brahches in $GIT_DIR/refs/heads/
somewhere, I never checkout those branches in my working tree.
I always stay in my branches to do my work.  I may "diff"
against them to see where I am.  Of course I would "resolve"
with them when I feel I am ready.

So, assumes that "linus" short-hand is set up to update
$GIT_DIR/refs/heads/linus with the foreign branch head, the
above example would have been:

    $ git checkout master && git pull linus
    : examine diffs and be convinced what Linus does is always right.

If my "master" branch is not ready to merge from Linus but I
want to get a feel of what is coming, I would instead do:

    : while staying on my master branch
    $ git fetch linus
    $ gitk linus master ;# or git show-branch linus master

and later when my branch is ready, I would merge it into my
master:

    : still staying on my master branch
    $ git pull . linus

If you did the pull into your master but it turns out that the
merge result is too messy, you could always reset (back to the
first example):

    $ git checkout master && git pull linus
    : if merge failed...
    $ git reset --hard master

    $ git checkout master && git pull linus
    : merge succeeds, but I realize that my tree was not quite
    : ready to merge from linus -- going back to pre-merge state
    $ git reset --hard master^1


The rules for updating $GIT_DIR/refs/ when fetch happens have
been extended and clarified in 0.99.5 quite a bit.

To set up "linus" short-hand to be updated with "master" branch
head from Linus, you would do one of the following:

  * Using old style shorthand

    $ echo >$GIT_DIR/branches/linus \
    http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
    $ git fetch linus

  * Using new style shorthand

    $ cat >$GIT_DIR/remotes/linus \
    URL: http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
    Pull: master:linus
    $ git fetch linus

  * From the command line, having either branches/linus or
    remotes/linus from the above two examples:

    $ git fetch linus master:linus

To set up "linus" short-hand _not_ to update any local branch
head (i.e. you only use "pull" to update your local branch,
which can be named "linus" branch), you would do one of the
following:

  * Using old style shorthand

    There is no way to do this using old style shorthand without
    an explicit command line <refspec>.  See the "From the
    command line" example below how to do this.

  * Using new style shorthand

    $ cat >$GIT_DIR/remotes/linus \
    URL: http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
    Pull: master:
    $ git fetch linus

  * From the command line, having either branches/linus or
    remotes/linus from the above two examples:

    $ git fetch linus master:

RE: cache status after git pull

[Luck, Tony]

>To set up "linus" short-hand to be updated with "master" branch
>head from Linus, you would do one of the following:
>
>  * Using new style shorthand
>
>    $ cat >$GIT_DIR/remotes/linus \
>    URL: http://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git/
>    Pull: master:linus
>    $ git fetch linus

This sounds like it does what I want ... so I'd like to confirm
before wiring this into my scripts and fingers.

What I want is to get the latest from kernel.org...linus...master
and update my .refs/heads/linus with the new SHA1.

I'd like to be able to do that without touching what is in my
index, and without changing the state of any checked out files.

If that is what the above does, then you are my hero for today :-)

-Tony

[PATCH] Fix pulling into the same branch.

[Junio C Hamano]

Earlier, I said:

    Subject: Re: cache status after git pull
    Date: Thu, 25 Aug 2005 13:26:07 -0700
    Message-ID: <7v4q9dvk4g.fsf@assigned-by-dhcp.cox.net>

    > 1) Updated my "linus" branch:
    >
    >   $ git checkout linus && git pull linus

    The second command, "git pull linus", would internally run "git
    fetch linus".  It depends on how your shorthand "linus" is
    defined, but if it is set to update (either overwrite or
    fast-forward) the "linus" branch head, then your HEAD pointer
    would be updated without updating the index and working tree.
    This is bad because now you are telling git that your working
    tree is based on updated "linus" branch head, and what you
    _could_ commit on top of it is the same thing as what old
    "linus" branch head commit used to have.  That's why "git
    status" output shows the minefield.

    If I keep copies of foreign brahches in $GIT_DIR/refs/heads/
    somewhere, I never checkout those branches in my working tree.
    I always stay in my branches to do my work.  I may "diff"
    against them to see where I am.  Of course I would "resolve"
    with them when I feel I am ready.

    So, assuming that "linus" short-hand is set up to update
    $GIT_DIR/refs/heads/linus with the foreign branch head, the
    above example would have been:

        $ git checkout master && git pull linus
        : examine diffs and be convinced what Linus does is always right.

This "do not pull into the branch you are on" is probably a good
advice as a workaround, but there is no fundamental reason to
forbid it.  Worse, there is no machinery to even warn about the
situation right now.

This patch is to show my current thinking.  Please let me know
what you think.

-jc

------------
When the "git pull" command updates the branch head you are
currently on, before doing anything else, first update your
index file and the working tree contents to that of the new
branch head.  Otherwise, the later resolving steps would think
your index file is attempting to revert the change between the
original head commit and the updated head commit.

It uses two-tree fast-forward form of "read-tree -m -u" to
prevent losing whatever local changes you may have in the
working tree to do this update.  I think this would at least
make things safer (a lot safer), and prevent mistakes.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 git-pull-script |   18 ++++++++++++++++++
 1 files changed, 18 insertions(+), 0 deletions(-)

10f624a7743fc1e29277de9875dbd6a882f2d3b3
diff --git a/git-pull-script b/git-pull-script
--- a/git-pull-script
+++ b/git-pull-script
@@ -5,7 +5,25 @@
 # Fetch one or more remote refs and merge it/them into the current HEAD.
 
 . git-sh-setup-script || die "Not a git archive"
+
+orig_head=$(cat "$GIT_DIR/HEAD") || die "Pulling into a black hole?"
 git-fetch-script "$@" || exit 1
+
+curr_head=$(cat "$GIT_DIR/HEAD")
+if test "$curr_head" != "$orig_head"
+then
+	# The fetch involved updating the current branch.
+
+	# The working tree and the index file is still based on the
+	# $orig_head commit, but we are merging into $curr_head.
+	# First update the working tree to match $curr_head.
+
+	echo >&2 "Warning: fetch updated the current branch head."
+	echo >&2 "Warning: fast forwarding your working tree."
+	git-read-tree -u -m "$orig_head" "$curr_head" ||
+		die "You need to first update your working tree."
+fi
+
 merge_head=$(sed -e 's/	.*//' "$GIT_DIR"/FETCH_HEAD | tr '\012' ' ')
 merge_name=$(sed -e 's/^[0-9a-f]*	//' "$GIT_DIR"/FETCH_HEAD |
 	 tr '\012' ' ')

Re: cache status after git pull

[Junio C Hamano]

"Luck, Tony" <tony.luck@intel.com> writes:

> What I want is to get the latest from kernel.org...linus...master
> and update my .refs/heads/linus with the new SHA1.
>
> I'd like to be able to do that without touching what is in my
> index, and without changing the state of any checked out files.
>
> If that is what the above does,...

At least that is how "git fetch <repo> <src>:<dst>" is designed
to work, and "Pull: <src>:<dst>" in a remotes/ shorthand is
designed to reduce typing while doing so.  Please bug me if
things do not work as advertised.

Re: [PATCH] Fix pulling into the same branch.

[Johannes Schindelin]

Hi,

On Thu, 25 Aug 2005, Junio C Hamano wrote:

> This patch is to show my current thinking.  Please let me know
> what you think.

I like it. As Linus stated, the index originally had a different role from 
what it has now, so it really should be an internal git thing, i.e. the 
git user should not expect the index not to change when pulling.

Ciao,
Dscho

Re: [PATCH] Fix pulling into the same branch.

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> I like it. As Linus stated, the index originally had a different role from 
> what it has now, so it really should be an internal git thing, i.e. the 
> git user should not expect the index not to change when pulling.

Actually the issue and the way patch addresses it is not limited
to the index file.  The problem is that the relashionship
between your working tree and the $GIT_DIR/HEAD commit changes
unexpectedly (from the POV of the working tree) when the ref
being updated by the "push in reverse" machinery happens to be
the head of the branch you are currently on.  The working tree
is supposed to be based on $GIT_DIR/HEAD, but after the "push in
reverse".  This fix is about fast forwarding the working tree,
including the checked out files on the filesystem, to the
updated head when it happens.

I am tempted to move this logic to "git fetch" instead, because
it has the same issue.  Tony's "linus" branch example has been
updated to do a "git fetch" instead of "git pull" from the
earlier description in his howto, but if he happens to be on the
"linus" branch, he would still have this same problem.

Of course, doing this in "git fetch" needs to be done a bit more
carefully than in "git pull", because by definition "pull" is a
fetch followed by merging into the working tree, implying that
the repository has an associated working tree.  But "fetch" by
itself can be used for a naked repository [*1*], and we should
not unconditionally attempt to update the working tree that may
not exist.

I am planning to use the lack of index file for this detection,
but I've seen some people made an rsync copy of their private
repository as a whole to publish, so this heuristic may
probabaly break.  Maybe the lack of "$GIT_DIR/index" and
$GIT_DIR not ending with "/.git" and not being ".git"?  I am
undecided.

In the meantime, warning the user about the issue and suggesting
how to do the fast-forwarding of the working tree himself in the
warning message might be the safest and the most sensible thing
to do.


[Footnote]

*1* Do we want a term in the glossary to mean repositories like
linux-2.6.git and git.git that do not have associated working
trees?