Re: What's cooking in git.git (Aug 2018, #04; Fri, 17)

[Christian Couder <christian.couder@gmail.com>]

On Sat, Aug 18, 2018 at 1:34 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Sat, Aug 18 2018, Christian Couder wrote:

> > FYI this has been requested from GitLab by Drupal (as well as others)
> > see https://www.drupal.org/drupalorg/blog/developer-tools-initiative-part-5-gitlab-partnership
> > which contains:
> >
> > "The timeline for Phase 2 is dependent on GitLab’s resolution of a
> > diskspace deduplication issue, which they have committed to on our
> > behalf: https://gitlab.com/gitlab-org/gitlab-ce/issues/23029"
>
> This is not a critique of the delta islands feature, just something I'm
> curious about.
>
> Why is Drupal blocked on something like delta-islands? The blog post
> mentions they have 45k projects, which can be browsed at
> https://cgit.drupalcode.org
>
> Almost all of those are completely independent projects, so they
> wouldn't benefit from delta islands, and I'd guess >98% are of them are
> in an obscure long tail and probably won't have even a single fork.
>
> That leaves forks of say drupal.git, which is ~150MB, the mirror on
> GitHub has 1500 forks: https://github.com/drupal Even if there were 5000
> forks of that that would be 750G of disk space.
>
> So accounting for backups, me being off by a lot etc. let's say that's
> 5TB. That's relatively cheap today. Are they really just holding up
> their GitLab migration plans to save something on the order of that disk
> space, or have I missed something here?

I am not sure why. I haven't been in touch directly with Drupal people
and I haven't followed all the discussions with them.

When I discussed this topic with someone responsible for another
significant open source project. He told me that they don't want forks
at all on their self hosted GitLab instance because of the disk space
and management burden that would come with them. But they would like
people to fork on a separate GitLab instance like gitlab.com or
github.com (but then be able to send merge/pull requests to their self
hosted GitLab instance) because they know that developers like to have
their own fork :-)

So my guess is that Drupal people are also afraid of the possible disk
space burden, even if your numbers seem to say that they shouldn't.

> Again, not a critique of delta-islands, because it's most certainly
> useful for the likes of github/gitlab, but I wonder if for this
> particular problem it wouldn't be more straightforward of a solution for
> GitLab to allow anyone to push to
> refs/for-merge/<their-username>/<some-name-they-pick> on any
> repository. Then they could open a MR for an existing branch in the repo
> (which GitLab already supports).

My guess is that people are used to forks on GitHub and they like them
and want to have the same thing and same workflow on GitLab.

The delta islands documentation though says that it's possible to use
git namespaces along with delta islands, and if forks are indeed
implemented as namespaces, it will be kind of similar in the GitLab
internals as what you suggest. There are still discussions by the way
in the GitLab issue referenced above about whether it's better for
GitLab to use git namespaces or alternates to implement deduplication
in forks (along with delta islands).