From: Kevin Wern <kevin.m.wern@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: Kevin Wern <kevin.m.wern@gmail.com>, git@vger.kernel.org
Subject: Re: [PATCH 07/11] Resumable clone: add resumable download to http/curl
Date: Wed, 28 Sep 2016 02:41:56 -0400 [thread overview]
Message-ID: <20160928064156.GG3762@kwern-HP-Pavilion-dv5-Notebook-PC> (raw)
In-Reply-To: <xmqqfuozihuv.fsf@gitster.mtv.corp.google.com>
On Fri, Sep 16, 2016 at 03:45:44PM -0700, Junio C Hamano wrote:
> > @@ -1136,7 +1138,10 @@ static int handle_curl_result(struct slot_results *results)
> > curl_easy_strerror(results->curl_result),
> > sizeof(curl_errorstr));
> > #endif
> > - return HTTP_ERROR;
> > + if (results->http_code >= 400)
> > + return HTTP_ERROR;
> > + else
> > + return HTTP_ERROR_RESUMABLE;
> > }
> > }
>
> Hmm, is "anything below 400" a good definition of resumable errors?
Definitely a rough definition, but I think it covers the majority of cases. I
based this on the fact that my example resumable errors (disconnections during
download) either had results->http_code set to 206 or 0, and that http errors
generally indicate some underlying issue that can't be solved with a repeat
request (there are plenty of exceptions, such as 503 or 504 errors that happen
intermittently). I thought that was enough to be a placeholder in the model. I
did consider something like "has (the file changed size or http_code been
206, some indication of success happened) in the last [x] attempts," but I
question the precision of something like that, especially before exploring
other tried-and-true solutions.
This patch is definitely the sketchiest part of my series, IMO. The concept
isn't fleshed out enough, even theoretically--containing only the element of
retrying with some very basic components to those decisions. I think that this
will ultimately require fishing through other examples of resumable downloads
(which was very painful to read when I tried). I know curl has a --retry option,
but doesn't include it in libcurl...harumph. I'll probably try to read that
implementation and chromium's resumable download code again.
I'll get back to you with a better plan here once I read through a few
examples.
- Kevin
next prev parent reply other threads:[~2016-09-28 6:42 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-16 0:12 [PATCH 00/11] Resumable clone Kevin Wern
2016-09-16 0:12 ` [PATCH 01/11] Resumable clone: create service git-prime-clone Kevin Wern
2016-09-16 20:53 ` Junio C Hamano
2016-09-28 4:40 ` Kevin Wern
2016-09-16 0:12 ` [PATCH 02/11] Resumable clone: add prime-clone endpoints Kevin Wern
2016-09-19 13:15 ` Duy Nguyen
2016-09-28 4:43 ` Kevin Wern
2016-09-16 0:12 ` [PATCH 03/11] pkt-line: create gentle packet_read_line functions Kevin Wern
2016-09-16 22:17 ` Junio C Hamano
2016-09-28 4:42 ` Kevin Wern
2016-09-16 0:12 ` [PATCH 04/11] Resumable clone: add prime-clone to remote-curl Kevin Wern
2016-09-19 13:52 ` Duy Nguyen
2016-09-28 6:45 ` Kevin Wern
2016-09-16 0:12 ` [PATCH 05/11] Resumable clone: add output parsing to connect.c Kevin Wern
2016-09-16 0:12 ` [PATCH 06/11] Resumable clone: implement transport_prime_clone Kevin Wern
2016-09-16 0:12 ` [PATCH 07/11] Resumable clone: add resumable download to http/curl Kevin Wern
2016-09-16 22:45 ` Junio C Hamano
2016-09-28 6:41 ` Kevin Wern [this message]
2016-09-16 0:12 ` [PATCH 08/11] Resumable clone: create transport_download_primer Kevin Wern
2016-09-16 0:12 ` [PATCH 09/11] path: add resumable marker Kevin Wern
2016-09-19 13:24 ` Duy Nguyen
2016-09-16 0:12 ` [PATCH 10/11] run command: add RUN_COMMAND_NO_STDOUT Kevin Wern
2016-09-16 23:07 ` Junio C Hamano
2016-09-18 19:22 ` Johannes Schindelin
2016-09-28 4:46 ` Kevin Wern
2016-09-28 17:54 ` Junio C Hamano
2016-09-28 18:06 ` Kevin Wern
2016-09-16 0:12 ` [PATCH 11/11] Resumable clone: implement primer logic in git-clone Kevin Wern
2016-09-16 23:32 ` Junio C Hamano
2016-09-28 5:49 ` Kevin Wern
2016-09-19 14:04 ` Duy Nguyen
2016-09-19 17:16 ` Junio C Hamano
2016-09-28 4:44 ` Kevin Wern
2016-09-16 20:47 ` [PATCH 00/11] Resumable clone Junio C Hamano
2016-09-27 21:51 ` Eric Wong
2016-09-27 22:07 ` Junio C Hamano
2016-09-28 17:32 ` Junio C Hamano
2016-09-28 18:22 ` Junio C Hamano
2016-09-28 20:46 ` Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160928064156.GG3762@kwern-HP-Pavilion-dv5-Notebook-PC \
--to=kevin.m.wern@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).