Re: Is offloading to GPU a worthwhile feature?

[Stefan Beller <sbeller@google.com>]

On Tue, Feb 27, 2018 at 12:52 PM, Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
> compression offload

Currently there is a series under review that introduces a commit graph
file[1], which would allow to not need decompressing objects for a rev walk, but
have the walking information as-needed on disk.

Once walking (as part of negotiation) is done,
we'd have to pack a pack file to return to the client,
which maybe can be improved by GPU acceleration[2].

Though once upon a time Junio had proposed to change
this part of the protocol as well. Instead of having a packfile
catered to a specific user/request, the server would store
multiple pack files, which are temporally sorted, e.g.
one "old" packfile containing everything that is roughly older
than 4 weeks ago, then a "medium pack file" that is up to last
weekend, and then a "new" pack that is just this weeks work,
followed by the on-demand pack that is just the latest
generated on the fly.

The server would just dump these different packfiles
(concatenated?) at the user, and would need to refresh
its packfiles occasionally every week or so.

[1] https://public-inbox.org/git/1519698787-190494-1-git-send-email-dstolee@microsoft.com/
[2] http://on-demand.gputechconf.com/gtc/2014/presentations/S4459-parallel-lossless-compression-using-gpus.pdf


> I realize this would be silly amounts of work. But, if it's worth it,
> perhaps we can benefit from all the GPU computation libs written for
> cryptocoin mining and use them for something good. :)

Currently there is work being done on "protocol v2"[3], which
is also motivated by the desire to have easy extensibility in the protocol,
so if you want to put in a cryptocoin-secret-handshake [into the protocol]
that improves the cost of compute or the bandwidth required for your
typical use case, it will be possible to do so with ease.

[3] https://public-inbox.org/git/20180207011312.189834-1-bmwill@google.com/

I wonder if the bitmap code can be sped up using GPUs. Every once in a while
the discussion floats up bloom filters or inverse bloom filters for
the negotiation
part, and a quick search shows that those could also be sped up using GPUs.

Stefan