[PATCH v2] travis-ci: retry if Git for Windows CI returns HTTP error 502 or 503

[Lars Schneider <larsxschneider@gmail.com>]

The Git for Windows CI web app sometimes returns HTTP errors of
"502 bad gateway" or "503 service unavailable" [1]. We also need to
check the HTTP content because the GfW web app seems to pass through
(error) results from other Azure calls with HTTP code 200.
Wait a little and retry the request if this happens.

[1] https://docs.microsoft.com/en-in/azure/app-service-web/app-service-web-troubleshoot-http-502-http-503

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---

Hi Junio,

I can't really test this as my TravisCI account does not have the
extended timeout and I am unable to reproduce the error.

It would be great if we could test this is a little bit in pu.

Thanks,
Lars

Notes:
    Base Ref: next
    Web-Diff: https://github.com/larsxschneider/git/commit/af0f0f0eb8
    Checkout: git fetch https://github.com/larsxschneider/git travisci/win-retry-v2 && git checkout af0f0f0eb8

    Interdiff (v1..v2):

    diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh
    index 7a9aa9c6a7..3e5a0abee0 100755
    --- a/ci/run-windows-build.sh
    +++ b/ci/run-windows-build.sh
    @@ -14,26 +14,33 @@ COMMIT=$2

     gfwci () {
     	local CURL_ERROR_CODE HTTP_CODE
    -	exec 3>&1
    +	CONTENT_FILE=$(mktemp -t "git-windows-ci-XXXXXX")
     	while test -z $HTTP_CODE
     	do
     	HTTP_CODE=$(curl \
     		-H "Authentication: Bearer $GFW_CI_TOKEN" \
     		--silent --retry 5 --write-out '%{HTTP_CODE}' \
    -		--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >cat >&3) \
    +		--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >$CONTENT_FILE) \
     		"https://git-for-windows-ci.azurewebsites.net/api/TestNow?$1" \
     	)
     	CURL_ERROR_CODE=$?
     		# The GfW CI web app sometimes returns HTTP errors of
     		# "502 bad gateway" or "503 service unavailable".
    -		# Wait a little and retry if it happens. More info:
    +		# We also need to check the HTTP content because the GfW web
    +		# app seems to pass through (error) results from other Azure
    +		# calls with HTTP code 200.
    +		# Wait a little and retry if we detect this error. More info:
     		# https://docs.microsoft.com/en-in/azure/app-service-web/app-service-web-troubleshoot-http-502-http-503
    -		if test $HTTP_CODE -eq 502 || test $HTTP_CODE -eq 503
    +		if test $HTTP_CODE -eq 502 ||
    +		   test $HTTP_CODE -eq 503 ||
    +		   grep "502 - Web server received an invalid response" $CONTENT_FILE >/dev/null
     		then
     			sleep 10
     			HTTP_CODE=
     		fi
     	done
    +	cat $CONTENT_FILE
    +	rm $CONTENT_FILE
     	if test $CURL_ERROR_CODE -ne 0
     	then
     		return $CURL_ERROR_CODE

    \0

 ci/run-windows-build.sh | 23 +++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/ci/run-windows-build.sh b/ci/run-windows-build.sh
index e043440799..3e5a0abee0 100755
--- a/ci/run-windows-build.sh
+++ b/ci/run-windows-build.sh
@@ -14,14 +14,33 @@ COMMIT=$2

 gfwci () {
 	local CURL_ERROR_CODE HTTP_CODE
-	exec 3>&1
+	CONTENT_FILE=$(mktemp -t "git-windows-ci-XXXXXX")
+	while test -z $HTTP_CODE
+	do
 	HTTP_CODE=$(curl \
 		-H "Authentication: Bearer $GFW_CI_TOKEN" \
 		--silent --retry 5 --write-out '%{HTTP_CODE}' \
-		--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >cat >&3) \
+		--output >(sed "$(printf '1s/^\xef\xbb\xbf//')" >$CONTENT_FILE) \
 		"https://git-for-windows-ci.azurewebsites.net/api/TestNow?$1" \
 	)
 	CURL_ERROR_CODE=$?
+		# The GfW CI web app sometimes returns HTTP errors of
+		# "502 bad gateway" or "503 service unavailable".
+		# We also need to check the HTTP content because the GfW web
+		# app seems to pass through (error) results from other Azure
+		# calls with HTTP code 200.
+		# Wait a little and retry if we detect this error. More info:
+		# https://docs.microsoft.com/en-in/azure/app-service-web/app-service-web-troubleshoot-http-502-http-503
+		if test $HTTP_CODE -eq 502 ||
+		   test $HTTP_CODE -eq 503 ||
+		   grep "502 - Web server received an invalid response" $CONTENT_FILE >/dev/null
+		then
+			sleep 10
+			HTTP_CODE=
+		fi
+	done
+	cat $CONTENT_FILE
+	rm $CONTENT_FILE
 	if test $CURL_ERROR_CODE -ne 0
 	then
 		return $CURL_ERROR_CODE

base-commit: 1ea7e62026c5dde4d8be80b2544696fc6aa70121
--
2.12.2