Re: clone hang prevention / timeout?

[Eric Wong <normalperson@yhbt.net>]

Jason Vas Dias <jason.vas.dias@gmail.com> wrote:
> Thanks very much Eric & Jeff for your reply .
> 
> Personally, I would recommend setting the SO_RECVTIMEO for GIT server
> sockets to a fixed default (eg. 5mins) , settable by a
> '--receive-timeout'   argument or configuration parameter .

(apologies for the delay, I thought I replied earlier :x)

SO_RCVTIMEO only triggers EAGAIN, and AFAIK the git read/write
wrappers are used to transparently retry on EAGAIN...  So it's
not so simple as doing a single setsockopt.

> The problem I was trying to overcome was cloning all the repositories under
> https://anongit.freedesktop.org/xorg/* .
> 
> About 4 git clones would succeed in succession, but then typically the 5th
> would hang in read() forever - I left one such hung 'git clone' for nearly an
> hour and it had not progressed or timed out . I tried inserting a delay of
> up to 30 seconds between clones, but this did not help.

Are you in contact with any of the admins of that server to
help?  Is the problematic repo any larger or in any way
stranger than the others?

> Maybe freedesktop.org's GIT server is too overloaded and they have
> to resort to disabling 1 out of 5 GIT successive clone operations from
> same connection or something.

Anyways I've been thinking about overloaded git servers, lately.
Pack generation on big repos is painful, and having lots of slow
clients can tie up server memory.  So maybe an HTTP server
which can switch between dumb and smart operation depending on
load could be useful for the resource-constrained.

> Here is my solution, in case anyone else needs it :

It'd be nice to get an strace to know where in the clone process
it hangs to help the admin figure out how far things got.

And please don't top-post, it's a waste of resources.