Re: [RFC/PATCH v3 00/16] Add initial experimental external ODB support

[Christian Couder <christian.couder@gmail.com>]

On Wed, Nov 30, 2016 at 11:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Christian Couder <christian.couder@gmail.com> writes:
>
>> For now there should be one odb ref per blob. Each ref name should be
>> refs/odbs/<odbname>/<sha1> where <sha1> is the sha1 of the blob stored
>> in the external odb named <odbname>.
>>
>> These odb refs should all point to a blob that should be stored in the
>> Git repository and contain information about the blob stored in the
>> external odb. This information can be specific to the external odb.
>> The repos can then share this information using commands like:
>>
>> `git fetch origin "refs/odbs/<odbname>/*:refs/odbs/<odbname>/*"`
>
> Unless this is designed to serve only a handful of blobs, I cannot
> see how this design would scale successfully.  I notice you wrote
> "For now" at the beginning, but what is the envisioned way this will
> evolve in the future?

In general I think that having a lot of refs is really a big problem
right now in Git as many big organizations using Git are facing this
problem in one form or another.
So I think that support for a big number of refs is a separate and
important problem that should and hopefully will be solved.

My preferred way to solve it would be with something like Shawn's
RefTree. I think it would also help regarding other problems like
speeding up git protocol, tracking patch series (see git-series
discussions), tools like https://www.softwareheritage.org/, ...

If the "big number of refs" problem is not solved and many refs in
refs/odbs/<odbname>/ is a problem, it's possible to have just one ref
in refs/odbs/<odbname>/ that points to a blob that contains a list
(maybe a json list with information attached to each item) of the
blobs stored in the external odb. Though I think it would be more
complex to edit/parse this list than to deal with many refs in
refs/odbs/<odbname>/.