Re: [PATCH 07/11] Resumable clone: add resumable download to http/curl

[Kevin Wern <kevin.m.wern@gmail.com>]

On Fri, Sep 16, 2016 at 03:45:44PM -0700, Junio C Hamano wrote:
> > @@ -1136,7 +1138,10 @@ static int handle_curl_result(struct slot_results *results)
> >  				curl_easy_strerror(results->curl_result),
> >  				sizeof(curl_errorstr));
> >  #endif
> > -		return HTTP_ERROR;
> > +		if (results->http_code >= 400)
> > +			return HTTP_ERROR;
> > +		else
> > +			return HTTP_ERROR_RESUMABLE;
> >  	}
> >  }
> 
> Hmm, is "anything below 400" a good definition of resumable errors?

Definitely a rough definition, but I think it covers the majority of cases. I
based this on the fact that my example resumable errors (disconnections during
download) either had results->http_code set to 206 or 0, and that http errors
generally indicate some underlying issue that can't be solved with a repeat
request (there are plenty of exceptions, such as 503 or 504 errors that happen
intermittently). I thought that was enough to be a placeholder in the model. I
did consider something like "has (the file changed size or http_code been
206, some indication of success happened) in the last [x] attempts," but I
question the precision of something like that, especially before exploring
other tried-and-true solutions.

This patch is definitely the sketchiest part of my series, IMO. The concept
isn't fleshed out enough, even theoretically--containing only the element of
retrying with some very basic components to those decisions. I think that this
will ultimately require fishing through other examples of resumable downloads
(which was very painful to read when I tried). I know curl has a --retry option,
but doesn't include it in libcurl...harumph. I'll probably try to read that
implementation and chromium's resumable download code again.

I'll get back to you with a better plan here once I read through a few
examples.

- Kevin