git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / Atom feed
From: Josh Triplett <josh@joshtriplett.org>
To: Jeff King <peff@peff.net>
Cc: git@vger.kernel.org
Subject: Re: upload-pack/ls-remote: showing non-HEAD symbolic refs?
Date: Tue, 16 Aug 2016 10:34:44 -0700
Message-ID: <20160816173444.rpqlpsz2ognvlufy@x> (raw)
In-Reply-To: <20160816163145.mjc726til2daxl3d@sigill.intra.peff.net>

On Tue, Aug 16, 2016 at 12:31:45PM -0400, Jeff King wrote:
> On Tue, Aug 16, 2016 at 09:18:39AM -0700, Josh Triplett wrote:
> 
> > Commit 5e7dcad771cb873e278a0571b46910d7c32e2f6c in September 2013 added
> > support to upload-pack to show the symbolic target of non-HEAD symbolic
> > refs.  However, commit d007dbf7d6d647dbcf0f357545f43f36dec46f3b in
> > November 2013 reverted that, because it used a capability to transmit
> > the information, and capabilities have a limited size (limited by the
> > pkt-line format which can't send lines longer than 64k) and can't
> > transmit an arbitrary number of symrefs.
> > 
> > (Incidentally, couldn't the same problem occur if the HEAD points to a
> > long enough path to exceed 64k?
> 
> Yes. But it's a lot easier to say "your 64k symref is ridiculous; don't
> do that" than it is to say "oh, you happened to have a lot of symrefs in
> your repository, so we overflowed and failed".
> 
> Besides which, the whole protocol cannot handle refnames larger than
> 64k, so it's not a new problem.

Absolutely agreed.  I mentioned it only because if a server provided any
mechanism to send it symbolic ref targets, this could lead to a
situation where you could push a repository that you could not
subsequently pull.  Depending on the protocols involved, that could
potentially require manual admin intervention, or could even result in a
DoS.

Not something that can arise with git itself, as send-pack/receive-pack
doesn't support sending symbolic ref targets, but I could imagine a
server doing so to allow setting HEAD.

> > I'd like to be able to see the targets of non-HEAD symbolic refs for a
> > repository (symbolic refs under refs/).  I'm interested in extending
> > upload-pack to expose those somehow.  What seems like a sensible format
> > to do so?
> > 
> > Would it make sense to advertise a new capability for symbolic ref
> > targets, which would allow the client to send back a dedicated request
> > for the targets of all symrefs?
> 
> It will definitely require a new capability. You cannot just send a
> "\0symref=..." trailer after each ref, because older clients treat
> multiple "\0" trailers as overwriting one another (so it essentially
> overwrites the old capabilities).
> 
> Sadly you cannot use a capability to fix that, because all of this
> happens before the client agrees to any capabilities (you can find
> discussion of a "v2" protocol on the list which solves this, but it's
> sort of languishing in the design phase).

As a potential 1.1 version, which could work in a backward-compatible
way with existing servers and no additional round-trip: what if, in the
smart HTTP protocol, the client advertised client capabilities with an
additional HTTP header (e.g.  "Git-Client-Caps: symrefs othershiny
featurenames"?  git-http-backend could then pass those capabilities to
git-upload-pack (--client-caps='...'), which could take them into
account in the initial response?

That wouldn't work as a single-pass approach for SSH, since the client
can't know if the server's upload-pack supports --client-caps, but it
would work for the smart HTTP protocol.

> So you are stuck introducing a new phase into the protocol, which is
> probably rather tricky (especially with the http protocol, which is very
> sensitive to extra round-trips). I guess the least-invasive way would be
> to communicate the desires in the "want" phase, and then have the server
> dump it out with the packfile. Like:
> 
>   - server claims "I support symref-wants" in the capability phase
> 
>   - during the negotiation phase, in addition to "want" and "have", the
>     client may send "symref <ref>" packets. Probably <ref> should be a
>     wildcard to avoid having to ask about each ref individually.
> 
>   - before outputting the packfile in the final phase, if any "symref"
>     wants were sent by the client, the server dumps a list of "symref
>     <from> <to>" packets, followed by a flush packet.
> 
> That should Just Work over the existing http protocol without requiring
> an extra request.

It'd require one extra request for git ls-remote, which normally doesn't
need the second round-trip, but that still seems reasonable.

  reply	other threads:[~2016-08-16 17:35 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-16 16:18 Josh Triplett
2016-08-16 16:31 ` Jeff King
2016-08-16 17:34   ` Josh Triplett [this message]
2016-08-16 18:28     ` Jeff King
2016-08-16 18:50       ` Stefan Beller
2016-08-16 20:34         ` Josh Triplett
2016-08-16 20:31       ` Josh Triplett
2016-08-16 20:54         ` Jeff King
2016-08-16 21:11           ` Josh Triplett
2016-08-16 21:15             ` Jeff King
2016-08-16 22:24               ` Josh Triplett
2016-08-19 13:47                 ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160816173444.rpqlpsz2ognvlufy@x \
    --to=josh@joshtriplett.org \
    --cc=git@vger.kernel.org \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git