Re: Question About Git V2 Protocol & SHA256

["brian m. carlson" <sandals@crustytoothpaste.net>]

[-- Attachment #1: Type: text/plain, Size: 5320 bytes --]

On 2020-09-30 at 12:20:06, Jeff King wrote:
> On Wed, Sep 30, 2020 at 02:19:25AM +0000, brian m. carlson wrote:
> 
> > If you have a suitably new Git, it will fail with an appropriate
> > message on both sides.  The client will complain that you are trying to
> > use a remote that uses the wrong hash algorithm and the server will
> > complain that you have failed to pass a suitable object-format
> > extension.  That will look like this:
> > 
> >   fatal: mismatched object format: server sha256; client sha1
> >   fatal: expected flush after ref listing
> 
> That makes sense, and the message isn't too bad.

Yeah, it seems reasonable.  If people like something else better, it can
be adjusted.

> > That's true if you're using protocol v2 at all, or a suitably new Git
> > with protocol v1, although the messages may differ slightly.  However,
> > if you're using an older Git with v1, then you'll get this:
> > 
> >   fatal: protocol error: unexpected '304c98b4860fa40b3e04f3e2e24db1a13ab959922a63209685813908c4fabc83 HEAD'
> >   fatal: the remote end hung up unexpectedly
> > 
> > That's because, as you pointed out, we don't get to the point where the
> > client fails to send the value.
> 
> That makes switching to sha256 on the server a bit of a flag day, then,
> doesn't it? Everybody with older Git will not be able to clone such a
> repo. That may be OK for some cases, but I suspect it is going to cause
> a delay in servers moving to sha256 themselves. Even if a project is
> ready to migrate, it may be hesitant to do so because of third parties
> outside of its control (whether external tools like CI, or just wanting
> to be open to cloning by random users).

Right, but as the code stands now, those people couldn't make use of a
SHA-256 repository anyway because there's no interoperability.

> On the other hand, any kind of hash negotiation would require on-the-fly
> conversion for that case (i.e., a sha256 server serving an older sha1
> client needs to produce the alternate-universe sha1 objects). That's
> expensive either in CPU or in disk space, and it may well be that
> third-party hosting sites would refuse to enable such an option anyway.

We will eventually have some sort of compatibility, and it's
theoretically possible that hosting could offer SHA-1 compatibility for
those repositories, even if that's only for protocol v1.  So the
repository could use SHA-1 for v1 and negotiate for v2.

The desired behavior for a large hoster like GitHub or GitLab may be to
force v2 only for SHA-256; that's a justifiable decision there, because
we know all clients will support it and we don't have to deal with
remote systems that are poorly configured.  So I wouldn't be opposed to
seeing an option that did that.  As mentioned elsewhere, though, it
isn't a 100% viable option everywhere.

> > Because v2 isn't implemented for pushes yet.  The testsuite does not
> > even come close to passing unless you have a fully functional remote
> > system.  That's why I implemented protocol support, even though it
> > wasn't originally planned.
> > 
> > That was mentioned in the cover letter of the series that introduced
> > protocol support.
> 
> Yeah, I think I misunderstood the direction there. I took the
> object-format capability to mean that the client and server would
> negotiate. But it really is the server saying "I'm going to speak this;
> you'd better understand it and be able to translate objects on your
> side".

Yes, at this point it is, and it will remain that way for v1.

> I did realize that getting the test suite to pass meant that we'd be
> passing sha256 hashes over the wire. But I had assumed that was part of
> the strategy to make everything work in a sha256 world, and that more
> interoperability stuff was coming on top. I.e., I thought our current
> state was less mature than you ended up implementing.

More interoperability is coming, but the stage 4 (SHA-256 only)
implementation was easier to implement, so we got that first.

The good news is that the compat code is less involved than I expected
because we providently made the decision to write every new loose object
in the repository using a single function.  The biggest problem is now
the testsuite because we try to stuff all manner of invalid and broken
object IDs in places and that doesn't work anymore because they can't be
mapped.  I'm chipping away at this problem while watching movies on the
weekend, so it's coming.

> > In retrospect, we probably should have sent "GIT_PROTOCOL=version=2" as
> > part of the shell command and let implementers deal with the fact that
> > their SSH servers would need to adequately parse and interpret shell
> > commands to function properly.  But it's probably too late to do that
> > now without bumping the protocol to v3.
> 
> That was definitely discussed, but rejected because of the
> interoperability problems. Not just for servers who actually want v2,
> but it would mean that the v2 probe would cause existing v1 servers to
> stop working. I.e., it would stop being safe to send v2 probes, which
> was the whole point of the scheme.

Yeah.  I guess we can't have everything we want in this case.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]