Re: [GSoC23] Working on project Idea from SOC 2011

[Derrick Stolee <derrickstolee@github.com>]

On 3/9/23 7:07 AM, Khalid Masum wrote:
> There is this SOC 2011 idea named "Resumable clone" here:
> 
> https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/SoC2011Ideas.html
> 
> ...
> Currently cloning a remote repository has to be done in one session.
> If the process fails or is aborted for any reason any already downloaded
> data is lost and one has to start from scratch.

> Goal: Allow Git to resume a cloning process that
> has been aborted for any reason.
> Languages: C

"for any reason" is going to be pretty difficult.
 
One direction that is relatively new in the Git project
(much newer than that project idea) is the bundle URI
standard, allowed by "git clone --bundle-uri=<X>". It
helps bootstrap clones by fetching bundle files and using
them to populate the object directory before finishing
the clone with an incremental fetch to the origin server.

Since the bundles are expected to be precomputed files,
it is much easier to use standard HTTP range queries to
download only the "missing" portion of the file from the
bundle server.

I think one thing that would need to change on the Git
client is the location of the temporary file being used
to store the bundle as it is downloaded. It currently
uses a random name, but if the name was a hash of the
URL, then it would be predictable and could restart the
download if the 'git clone' process was halted for any
reason. (Resuming a download due to a network error
noticed in-process is possibly simpler.)

This might be a more focused approach that is more
likely to have progress in a GSoC project.

That said, I don't have the capacity to be a mentor,
but I thought it worth mentioning this variant of the
project.

Thanks,
-Stolee