About connection resuming

About connection resuming

[chenzero]

Hello,
First, I am a user of git for about 2 years, I really appreciated you 
all to create this great useful software!
My encountered problem is:
sometimes, the repo is big and because my networking is not stable(or my 
network proxy has
some limitations), so, the clone will always fail.
I tried following ways to solve this, however, not much success.
1. clone depth 1
git clone --depth=1 https://..../repo.git
however, some repo even depth=1 might fail.(It seemed that my network 
proxy limit tcp connection.
if transfer over 50M, it will break)
2. download bundle file.
but no all git repo provide bundle files to download.

After some investment on the code, I think, perhaps,
if enhance the git-http-backend to support http header: Range, or 
Content-Range,
maybe it will enable connection resuming.
the problem of this way is: it needs to upgrade the current deployed 
"git-http-backend",
and maybe much code need to change including git-remote-http etc.

This is the very basic thought, and whether I should try other way ?
Thanks a lot!

Re: About connection resuming

[Stefan Beller]

On Tue, Feb 20, 2018 at 4:49 AM, chenzero <chenzero@netease.com> wrote:
> Hello,
> First, I am a user of git for about 2 years, I really appreciated you all to
> create this great useful software!

Hi, welcome to the mailing list!

> My encountered problem is:
> sometimes, the repo is big and because my networking is not stable(or my
> network proxy has
> some limitations), so, the clone will always fail.
> I tried following ways to solve this, however, not much success.
> 1. clone depth 1
> git clone --depth=1 https://..../repo.git
> however, some repo even depth=1 might fail.(It seemed that my network proxy
> limit tcp connection.
> if transfer over 50M, it will break)
> 2. download bundle file.
> but no all git repo provide bundle files to download.
>
> After some investment on the code, I think, perhaps,
> if enhance the git-http-backend to support http header: Range, or
> Content-Range,
> maybe it will enable connection resuming.

This thought has come up before, for example
https://public-inbox.org/git/1473984742-12516-1-git-send-email-kevin.m.wern@gmail.com/
(Or you can search for "resumable clone" in that mailing list archive,
there are more interesting discussions)

But the current problem is that the bytes of a clone are not stable,
as the packs are ordered racily IIUC: So if you clone the same repository
(with the same branch values), the observed communication over the wire
may differ in the part when the pack file is transferred as the packfile is
packed up in a multi threaded fashion, that has no clear order defined.

> the problem of this way is: it needs to upgrade the current deployed
> "git-http-backend",
> and maybe much code need to change including git-remote-http etc.
>
> This is the very basic thought, and whether I should try other way ?
> Thanks a lot!

I think a more promising approach is to have support for references
inside the packfile for transfer, this could become one of the features
of the new protocol that is being worked on
https://public-inbox.org/git/20180207011312.189834-1-bmwill@google.com/
I do not find a reference for the idea, but Jonathan (cc'd) laid it out
well once upon a time. Jonathan do you have a mailing list reference
for references in packfiles?

Thanks,
Stefan