remote-helpers: http-push protocol

remote-helpers: http-push protocol

[Sverre Rabbelier]

Heya,

I'm trying to figure out how the current http push protocol is
implemented. What I've been able to deduce on my own:

When fetching from the remote, the helper is called with 'list', after
which it will give back a list of refs, 'remote' style. That is, the
remote master branch will be listed as '<remote sha>
refs/heads/master'.
When pushing to the remote, the helper is called with 'list for-push',
it will again list all refs in the same manner (omitting 'HEAD' if the
smart protocol is used).

What happens next is the part that confuses me, a certain set of refs
is pushed, by calling the helper with 'push <refspec>'. I am unclear
how this set of refs is chosen, I see that only refs that have a
peer_ref are sent, unless mirror is also set (I don't think I
understand the significance)? After the push, the helper write a set
of 'ok/error' messages, sometimes for refs that were not asked to be
pushed in the first place, errors for those are ignored though (the
error parsing code ignores an error if the ref was not scheduled to be
pushed).

Can you clarify the behavior I described? How can I best hook into the
'refs that need to be pushed selection' in the 'export' command [0]?

[0] The export command currently tries to export all refs it gets from
push_refs, which seems to be everything that the helper got from the
'list' command. The only reason this has worked so far it seems is
that all branches listed by 'list' were present as local branches.

-- 
Cheers,

Sverre Rabbelier

Re: remote-helpers: http-push protocol

[Jonathan Nieder]

Hi,

Here's a know-nothing view (and I assume you have already done this
legwork; just summarizing for myself).  Following along in do_push().

Sverre Rabbelier wrote:

> When fetching from the remote, the helper is called with 'list',

get_ref_map() -> transport_get_remote_refs() ->
transport::get_refs_list()

[...]
> What happens next is the part that confuses me, a certain set of refs
> is pushed, by calling the helper with 'push <refspec>'.

The transport_push() codepath in transport.c.  For remote helpers, it
relies on the push_refs() method, calling:

 - transport::get_refs_list() to get the list of remote refs
 - get_local_heads() for the local refs
 - match_refs() to list remote refs matching the refspec/--all/
   --mirror
 - transport::push_refs() to push the list of refs chosen
 - transport_print_push_status() for a nice message
 - set_upstreams() to handle --set-upstream requests
 - transport_update_tracking_ref() to update the local remote-tracking
   refs.

> I am unclear
> how this set of refs is chosen

remote.c's match_refs() calls:

 - parse_push_refspec() to convert the refspec strings to
   structs refspec, separating src, dest, and +/*/no-dest-specified
   flags
 - match_explicit_refs() to build a linked list of struct ref
   explicitly requested with non-wildcard refspecs (the easy ones)
 - check_pattern_match() for each remote ref, to figure out which
   refspec, if any, it corresponds to.  If any:
    - Refs outside the refs/heads/ hierarchy are excluded (unless
      --mirror as used) when not named by a refspec, as you mentioned.
    - Where actual refspecs are concerned, though, they can match.
    - A local peer is determined with find_ref_by_name().
    - Enforces the "push matching" rule: do not push a new branch
      unless it was explicitly requested by naming it, --all, or
      --mirror.
 
> I see that only refs that have a
> peer_ref are sent, unless mirror is also set (I don't think I
> understand the significance)?

$ git log -S'"matching refs"; traditionally we pushed everything' remote.c
commit 098e711e6c0d123dff2f38d6b804ec632ad7dd78
Author: Junio C Hamano <gitster@pobox.com>
Date:   Sun Jul 1 19:00:08 2007 -0700

    "git-push $URL" without refspecs pushes only matching branches

    When "git push" is run without any refspec (neither on the
    command line nor in the config), we used to push "matching refs"
    in the sense that anything under refs/ hierarchy that exist on
    both ends were updated.  This used to be a sane default for
    publishing your repository to another back when we did not have
    refs/remotes/ hierarchy, but it does not make much sense these
    days.

    This changes the semantics to push only "matching branches".

So I suspect this is to ensure that with "[push] default = matching",

 $ git push

pushes branches and not remote-tracking refs, but

 $ git push --mirror

pushes everything.

> After the push, the helper write a set
> of 'ok/error' messages sometimes for refs that were not asked to be
> pushed in the first place

Part of transport::push_refs().  I think it only allows status
messages for refs that were actually pushed; otherwise it will print

	warning: helper reported unexpected status of <ref>

> Can you clarify the behavior I described? How can I best hook into the
> 'refs that need to be pushed selection' in the 'export' command [0]?

Hmm --- do you want some other hierarchy than refs/heads/* for the
"push matching" rule?

Thanks for the nice introduction.