Re: SHA256 support not experimental, or?

[Patrick Steinhardt <ps@pks.im>]

[-- Attachment #1: Type: text/plain, Size: 3981 bytes --]

On Fri, Jun 30, 2023 at 01:25:06PM +0200, Adam Majer wrote:
> On 6/30/23 11:31, Patrick Steinhardt wrote:
> > Indeed, supporting SHA256 is a major effort on our side at GitLab. Most
> > of the work isn't really adapting our production code, but it's rather
> > that tons of tests were written with seed repositories and hardcoded
> > object hashes. Converting all of that isn't all that hard in the general
> > case, but it's a tedious job.
> 
> Hi!
> 
> This actually reminds me of a funny story from my side.
> 
> Earlier this year, I was testing various frontends and how they would handle
> SHA256 repositories. All of them failed, not surprising. I even managed to
> lock myself out of Gitlab by importing a SHA256 private repo into my home
> project -- every time this project became visible, it would result in Error
> 500 from the UI. Today (few weeks ago), this appears to be fixed -- the UI
> is just broken, so you can't see anything in sha256 repository, but at least
> I was able to delete the project.

Yeah, thinks gradually start to work. It's kind of satisfying to see how
more and more things start to fall into place.

> The repository was correctly imported and I could clone from gitlab, so the
> problem is mostly "just" UI. :-)

The UI is a significantly broken right now, mostly because the request
routing logic still has a maximum object ID length of 40 characters
hardcoded. So indeed, most of the stuff in the UI doesn't work unless
you do a few changes in the frontend. I should probably just create the
merge request to fix these as I already have those changes available
locally anyway.

But there's other parts that are in the Gitaly backend that don't yet
work. There's some RPCs that parse object IDs, but still use the
hardcoded SHA1 hash. Updating them is trivial, but as mentioned updating
their tests is tedious work.

> The most likely frontend we'll use for our internal project is Gitea. The
> sha256 support is in progress
> 
> https://github.com/go-gitea/gitea/pull/23894
> 
> From the size of this patch, you can see how ingrained SHA1 assumption was.
> Most of the patch is just to remove the hardcoded elements, including
> hardcoded SHA1 empty-tree hashes and assumption that 20 bytes is enough to
> hold a hash. And I didn't even add sha256 test cases...

I guess most projects that started a long time a go made the same error
of taking SHA1 for granted, so they didn't bother writing neither the
production code nor the tests with swapping out the object format in
mind. I guess we've learned our lesson here, which also means that the
next transition (if there ever will be one) should go a lot faster as
the codebases should be prepared then.

> But I have to say that in at least one occasion, people are bringing up the
> experimental nature of git's sha256 support (per current wording) as reason
> not to make their tools sha256 compliant.

Yeah, it's this chicken-and-egg problem. Things are experimental as most
tools ain't got support, but because most things ain't got support we
never get any testers and thus are stuck in that state.

> > In any case I'm fully supportive of relaxing the current warning. Except
> > for the recently discussed edge case where cloning empty repositories
> > didn't create a SHA256 repository I have found the SHA256 code to be
> > stable and working as advertised. We should caution people that many
> > services will not work with SHA256 yet though.
> 
> That is exactly true. But this is also chicken-egg problem. Services are not
> adapted for sha256 repositories because there is simply no demand for them.
> Only when people will start using sha256 repos, will there be some demand
> generated.

Yup, and that is why I have been pushing for SHA256 support internally
at GitLab for quite a while -- our efforts here started almost exactly a
year ago, but has gained more steam in recent months.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]