[PATCH 0/3] Allow configuration of HTTP authentication method

[PATCH 0/3] Allow configuration of HTTP authentication method

[Simon.Richter]

From: Simon Richter <Simon.Richter@hogyros.de>

Hi,

this adds a configuration option to set the authentication method curl uses
when authenticating to a server.

The method is always configured, not just when a username is set, to allow
failing early if the server does not support the selected method;
otherwise, this mechanism is orthogonal to the proxy authentication method
handling, and I've liberally copied code from there.

This introduces http.authmethod and remote.<name>.authmethod configuration
options and an environment variable GIT_HTTP_AUTHMETHOD, with ascending
precedence.

There are three patches in this series, one that just renames a constant
list of options as it is used outside the proxy configuration scope now,
one that contains the main patch, and one I'm unsure about (hence no
Signed-Off-By yet) that allows empty user names during authentication.

The latter avoids surprises when people half-follow Microsoft's
documentation, which suggests that users configure a custom header
containing a hand-crafted Basic authentication string with an empty user
name. This is not strictly required by the "DevOps" server, any string will
do here, but simply pressing return on the username prompt will otherwise
fail to present the credentials at all, and give an error message
indicating that the given token is invalid.

I haven't investigated fully whether this is of any use outside the
interactive case, so the third patch is more of a request for comments.

With these changes, I can successfully authenticate to MS DevOps server
over HTTP using a Personal Access Token, without using the custom header
workaround[1], which allows me to use git-lfs (which in turn doesn't work
over ssh) from Jenkins with a limited token that is stored in the Jenkins
credential store, solving a problem for approximately tens of users[2].

   Simon

[1] https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate?view=azure-devops&tabs=Windows#use-a-pat

[2] https://stackoverflow.com/q/64800010

Simon Richter (3):
  Rename proxy_authmethods -> authmethods
  Add config option/env var to limit HTTP auth methods
  Allow empty user name in HTTP authentication

 Documentation/config/http.txt   | 19 ++++++++++++++
 Documentation/config/remote.txt |  4 +++
 http.c                          | 45 ++++++++++++++++++++++++++++-----
 remote.c                        |  4 +++
 remote.h                        |  3 +++
 5 files changed, 68 insertions(+), 7 deletions(-)

-- 
2.30.2

[PATCH 1/3] Rename proxy_authmethods -> authmethods

[Simon.Richter]

From: Simon Richter <Simon.Richter@hogyros.de>

Curl also allows specifying a list of acceptable auth methods for the
request itself, so this isn't specific to proxy authentication.

Signed-off-by: Simon Richter <Simon.Richter@hogyros.de>
---
 http.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/http.c b/http.c
index 229da4d148..318dc5daea 100644
--- a/http.c
+++ b/http.c
@@ -79,7 +79,7 @@ static int proxy_ssl_cert_password_required;
 static struct {
 	const char *name;
 	long curlauth_param;
-} proxy_authmethods[] = {
+} authmethods[] = {
 	{ "basic", CURLAUTH_BASIC },
 	{ "digest", CURLAUTH_DIGEST },
 	{ "negotiate", CURLAUTH_GSSNEGOTIATE },
@@ -470,14 +470,14 @@ static void init_curl_proxy_auth(CURL *result)
 
 	if (http_proxy_authmethod) {
 		int i;
-		for (i = 0; i < ARRAY_SIZE(proxy_authmethods); i++) {
-			if (!strcmp(http_proxy_authmethod, proxy_authmethods[i].name)) {
+		for (i = 0; i < ARRAY_SIZE(authmethods); i++) {
+			if (!strcmp(http_proxy_authmethod, authmethods[i].name)) {
 				curl_easy_setopt(result, CURLOPT_PROXYAUTH,
-						proxy_authmethods[i].curlauth_param);
+						authmethods[i].curlauth_param);
 				break;
 			}
 		}
-		if (i == ARRAY_SIZE(proxy_authmethods)) {
+		if (i == ARRAY_SIZE(authmethods)) {
 			warning("unsupported proxy authentication method %s: using anyauth",
 					http_proxy_authmethod);
 			curl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);
-- 
2.30.2

[PATCH 2/3] Add config option/env var to limit HTTP auth methods

[Simon.Richter]

From: Simon Richter <Simon.Richter@hogyros.de>

This allows forcing an authentication mechanism when the available
credentials do not match the automatically selected "best" mechanism.

For example, MS DevOps server supports both NTLM and Basic authentication,
but the NTLM backend is connected to the user database only and does not
accept Personal Access Tokens; curl however selects NTLM over Basic if both
are available.

Signed-off-by: Simon Richter <Simon.Richter@hogyros.de>
---
 Documentation/config/http.txt   | 19 +++++++++++++++++++
 Documentation/config/remote.txt |  4 ++++
 http.c                          | 33 ++++++++++++++++++++++++++++++++-
 remote.c                        |  4 ++++
 remote.h                        |  3 +++
 5 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/Documentation/config/http.txt b/Documentation/config/http.txt
index 7003661c0d..d9875afa4d 100644
--- a/Documentation/config/http.txt
+++ b/Documentation/config/http.txt
@@ -1,3 +1,22 @@
+http.authMethod::
+	Set the method with which to authenticate to the HTTP server, if
+	required. This can be overridden on a per-remote basis; see
+	`remote.<name>.authMethod`. Both can be overridden by the
+	`GIT_HTTP_AUTHMETHOD` environment variable.  Possible values are:
++
+--
+* `anyauth` - Automatically pick a suitable authentication method. It is
+  assumed that the server answers an unauthenticated request with a 401
+  status code and one or more WWW-Authenticate headers with supported
+  authentication methods. This is the default.
+* `basic` - HTTP Basic authentication
+* `digest` - HTTP Digest authentication; this prevents the password from being
+  transmitted to the server in clear text
+* `negotiate` - GSS-Negotiate authentication (compare the --negotiate option
+  of `curl(1)`)
+* `ntlm` - NTLM authentication (compare the --ntlm option of `curl(1)`)
+--
+
 http.proxy::
 	Override the HTTP proxy, normally configured using the 'http_proxy',
 	'https_proxy', and 'all_proxy' environment variables (see `curl(1)`). In
diff --git a/Documentation/config/remote.txt b/Documentation/config/remote.txt
index 0678b4bcfe..0f87234427 100644
--- a/Documentation/config/remote.txt
+++ b/Documentation/config/remote.txt
@@ -10,6 +10,10 @@ remote.<name>.url::
 remote.<name>.pushurl::
 	The push URL of a remote repository.  See linkgit:git-push[1].
 
+remote.<name>.authMethod::
+	For http and https remotes, the method to use for
+	authenticating against the server. See `http.authMethod`.
+
 remote.<name>.proxy::
 	For remotes that require curl (http, https and ftp), the URL to
 	the proxy to use for that remote.  Set to the empty string to
diff --git a/http.c b/http.c
index 318dc5daea..c5af90b1b8 100644
--- a/http.c
+++ b/http.c
@@ -108,6 +108,7 @@ static const char *curl_proxyuserpwd;
 static const char *curl_cookie_file;
 static int curl_save_cookies;
 struct credential http_auth = CREDENTIAL_INIT;
+static const char *http_authmethod;
 static int http_proactive_auth;
 static const char *user_agent;
 static int curl_empty_auth = -1;
@@ -356,6 +357,9 @@ static int http_options(const char *var, const char *value, void *cb)
 	if (!strcmp("http.useragent", var))
 		return git_config_string(&user_agent, var, value);
 
+	if (!strcmp("http.authmethod", var))
+		return git_config_string(&http_authmethod, var, value);
+
 	if (!strcmp("http.emptyauth", var)) {
 		if (value && !strcmp("auto", value))
 			curl_empty_auth = -1;
@@ -450,6 +454,27 @@ static void var_override(const char **var, char *value)
 	}
 }
 
+static void init_curl_http_auth_method(CURL *result)
+{
+	var_override(&http_authmethod, getenv("GIT_HTTP_AUTHMETHOD"));
+
+	if (http_authmethod) {
+		int i;
+		for (i = 0; i < ARRAY_SIZE(authmethods); i++) {
+			if (!strcmp(http_authmethod, authmethods[i].name)) {
+				http_auth_methods = authmethods[i].curlauth_param;
+				break;
+			}
+		}
+		if (i == ARRAY_SIZE(authmethods)) {
+			warning("unsupported authentication method %s: using anyauth",
+					http_authmethod);
+			http_auth_methods = CURLAUTH_ANY;
+		}
+	}
+	curl_easy_setopt(result, CURLOPT_HTTPAUTH, http_auth_methods);
+}
+
 static void set_proxyauth_name_password(CURL *result)
 {
 		curl_easy_setopt(result, CURLOPT_PROXYUSERNAME,
@@ -786,7 +811,7 @@ static CURL *get_curl_handle(void)
 #endif
 
 	curl_easy_setopt(result, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
-	curl_easy_setopt(result, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
+	init_curl_http_auth_method(result);
 
 #ifdef CURLGSSAPI_DELEGATION_FLAG
 	if (curl_deleg) {
@@ -1040,6 +1065,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	if (remote && remote->http_proxy)
 		curl_http_proxy = xstrdup(remote->http_proxy);
 
+	if (remote)
+		var_override(&http_authmethod, remote->http_authmethod);
+
 	if (remote)
 		var_override(&http_proxy_authmethod, remote->http_proxy_authmethod);
 
@@ -1504,6 +1532,9 @@ static int handle_curl_result(struct slot_results *results)
 			if (results->auth_avail) {
 				http_auth_methods &= results->auth_avail;
 				http_auth_methods_restricted = 1;
+				/* fail if no methods left */
+				if(http_auth_methods == 0)
+					return HTTP_NOAUTH;
 			}
 			return HTTP_REAUTH;
 		}
diff --git a/remote.c b/remote.c
index 42a4e7106e..dca7b82c9f 100644
--- a/remote.c
+++ b/remote.c
@@ -155,6 +155,7 @@ static void remote_clear(struct remote *remote)
 	FREE_AND_NULL(remote->pushurl);
 	free((char *)remote->receivepack);
 	free((char *)remote->uploadpack);
+	FREE_AND_NULL(remote->http_authmethod);
 	FREE_AND_NULL(remote->http_proxy);
 	FREE_AND_NULL(remote->http_proxy_authmethod);
 }
@@ -461,6 +462,9 @@ static int handle_config(const char *key, const char *value, void *cb)
 			remote->fetch_tags = -1;
 		else if (!strcmp(value, "--tags"))
 			remote->fetch_tags = 2;
+	} else if (!strcmp(subkey, "authmethod")) {
+		return git_config_string((const char **)&remote->http_authmethod,
+					 key, value);
 	} else if (!strcmp(subkey, "proxy")) {
 		return git_config_string((const char **)&remote->http_proxy,
 					 key, value);
diff --git a/remote.h b/remote.h
index 4a1209ae2c..c063d30356 100644
--- a/remote.h
+++ b/remote.h
@@ -105,6 +105,9 @@ struct remote {
 	const char *receivepack;
 	const char *uploadpack;
 
+	/* The method for authenticating against the (HTTP) server */
+	char *http_authmethod;
+
 	/* The proxy to use for curl (http, https, ftp, etc.) URLs. */
 	char *http_proxy;
 
-- 
2.30.2

[RFC PATCH 3/3] Allow empty user name in HTTP authentication

[Simon.Richter]

From: Simon Richter <Simon.Richter@hogyros.de>

When using a Personal Access Token in Microsoft DevOps server, the username
can be empty, so users might expect that pressing return on an username
prompt will work.
---
 http.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/http.c b/http.c
index c5af90b1b8..dc71fb75ea 100644
--- a/http.c
+++ b/http.c
@@ -433,7 +433,7 @@ static int curl_empty_auth_enabled(void)
 
 static void init_curl_http_auth(CURL *result)
 {
-	if (!http_auth.username || !*http_auth.username) {
+	if (!http_auth.username) {
 		if (curl_empty_auth_enabled())
 			curl_easy_setopt(result, CURLOPT_USERPWD, ":");
 		return;
-- 
2.30.2

Re: [PATCH 1/3] Rename proxy_authmethods -> authmethods

[Junio C Hamano]

Simon.Richter@hogyros.de writes:

> From: Simon Richter <Simon.Richter@hogyros.de>
>
> Curl also allows specifying a list of acceptable auth methods for the
> request itself, so this isn't specific to proxy authentication.

While that is true, given that it is ONLY used to sanity check the
http_proxy_authmethod variable and use CURLOPT_PROXYAUTH thing, the
above alone is not a good excuse to rename this array.

I haven't read the later patches, but I would imagine that this is
so that you'd create another consumer that is about authentication
method that is not for the proxyauth.  And if that is the case, the
proposed log message for this change should explicitly say so to
justify this change.

    We are about to reuse this table of authmethods to parse the
    non-proxy authentication in a later step in this series.  Let's
    rename it to just "authmethod[]".

appended as the second paragraph after the above might be sufficient.

Two things that comes to mind:

 * In general, an array whose elements are accessed individually is
   better named in singular, i.e. "type thing[]", not "type
   things[]" (an exception is when the most prevalent use of the
   array is to pass it as a whole to functions as a bag of things,
   instead of accessing individual element).  This is because it is
   more natural to see the zeroth thing to be spelled "thing[0]",
   and not "things[0]".  If we are renaming this array anyway, it
   may make sense to rename it to authmethod[].

 * If the reason why this rename is warranted is because there will
   be another user of this table that maps a string name to its
   corresponding CURLAUTH_* constant, it would probably make sense
   to extract a helper function out of this loop to do just that,
   something along the lines of ...

   static int parse_authmethod(const char *name, long *auth_param)
   {
       int i;

       for (i = 0; i < ARRAY_SIZE(authmethod); i++)
           if (!strcmp(name, authmethod[i].name)) {
		*auth_param = authmethod[i].curlauth_param;
                return i;
           }
       return -1;
   }

   Then the existing code can become

   if (http_proxy_authmethod) {
        long auth_param;

	if (parse_authmethod(http_proxy_authmethod, &auth_param) < 0) {
	    warning("unsupported ... %s: using anyauth", http_proxy_authmethod);
            auth_param = CURLAUTH_ANY;
	}
	curl_easy_setopt(result, CURLOPT_PROXYAUTH, auth_param);
   }

   It is probably OK to do so in the same patch as renaming of the
   table, but the focus of the step will then become "factor out
   parsing of authmethod from string to CURLAUTH_* constants" and
   the patch should be retitled accordingly.  Then you do not have
   to justify the rename of the table based on the future plan.

> Subject: Re: [PATCH 1/3] Rename proxy_authmethods -> authmethods

The title of a patch in this project follows certain convention.
cf. Documentation/SubmittingPatches.

    Subject: [PATCH 1/n] http: factor out parsing of authmethod

    In order to support CURLOPT_PROXYAUTH, there is a code to parse
    the name of an authentication method given as a string into one
    of the CURLAUTH_* constant.  The next step of this series wants
    to reuse the same parser to support CURLOPT_HTTPAUTH in a
    similar way.

    Factor out the loop into a separate helper function.  Since the
    table of authentication methods no longer is only for proxy
    authentication, drop "proxy" prefix from its name while we are
    at it.

or something like that, perhaps.

> Signed-off-by: Simon Richter <Simon.Richter@hogyros.de>
> ---
>  http.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/http.c b/http.c
> index 229da4d148..318dc5daea 100644
> --- a/http.c
> +++ b/http.c
> @@ -79,7 +79,7 @@ static int proxy_ssl_cert_password_required;
>  static struct {
>  	const char *name;
>  	long curlauth_param;
> -} proxy_authmethods[] = {
> +} authmethods[] = {
>  	{ "basic", CURLAUTH_BASIC },
>  	{ "digest", CURLAUTH_DIGEST },
>  	{ "negotiate", CURLAUTH_GSSNEGOTIATE },
> @@ -470,14 +470,14 @@ static void init_curl_proxy_auth(CURL *result)
>  
>  	if (http_proxy_authmethod) {
>  		int i;
> -		for (i = 0; i < ARRAY_SIZE(proxy_authmethods); i++) {
> -			if (!strcmp(http_proxy_authmethod, proxy_authmethods[i].name)) {
> +		for (i = 0; i < ARRAY_SIZE(authmethods); i++) {
> +			if (!strcmp(http_proxy_authmethod, authmethods[i].name)) {
>  				curl_easy_setopt(result, CURLOPT_PROXYAUTH,
> -						proxy_authmethods[i].curlauth_param);
> +						authmethods[i].curlauth_param);
>  				break;
>  			}
>  		}
> -		if (i == ARRAY_SIZE(proxy_authmethods)) {
> +		if (i == ARRAY_SIZE(authmethods)) {
>  			warning("unsupported proxy authentication method %s: using anyauth",
>  					http_proxy_authmethod);
>  			curl_easy_setopt(result, CURLOPT_PROXYAUTH, CURLAUTH_ANY);

Re: [PATCH 2/3] Add config option/env var to limit HTTP auth methods

[Junio C Hamano]

Simon.Richter@hogyros.de writes:

> +http.authMethod::
> +	Set the method with which to authenticate to the HTTP server, if
> +	required. This can be overridden on a per-remote basis; see
> +	`remote.<name>.authMethod`. Both can be overridden by the
> +	`GIT_HTTP_AUTHMETHOD` environment variable.  Possible values are:
> ++
> +--
> +* `anyauth` - Automatically pick a suitable authentication method. It is
> +  assumed that the server answers an unauthenticated request with a 401
> +  status code and one or more WWW-Authenticate headers with supported
> +  authentication methods. This is the default.
> +* `basic` - HTTP Basic authentication
> +* `digest` - HTTP Digest authentication; this prevents the password from being
> +  transmitted to the server in clear text
> +* `negotiate` - GSS-Negotiate authentication (compare the --negotiate option
> +  of `curl(1)`)
> +* `ntlm` - NTLM authentication (compare the --ntlm option of `curl(1)`)
> +--

The above makes sense.

Configuring this variable per URL, just like all other variables in
"http.*" namespace, we should be able to use the "http.<url>.*"
mechanism that the users are already familiar with.

> diff --git a/Documentation/config/remote.txt b/Documentation/config/remote.txt
> index 0678b4bcfe..0f87234427 100644
> --- a/Documentation/config/remote.txt
> +++ b/Documentation/config/remote.txt
> @@ -10,6 +10,10 @@ remote.<name>.url::
>  remote.<name>.pushurl::
>  	The push URL of a remote repository.  See linkgit:git-push[1].
>  
> +remote.<name>.authMethod::
> +	For http and https remotes, the method to use for
> +	authenticating against the server. See `http.authMethod`.

IOW, this looks out of place.

Re: [RFC PATCH 3/3] Allow empty user name in HTTP authentication

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

On 2022-05-13 at 07:04:16, Simon.Richter@hogyros.de wrote:
> From: Simon Richter <Simon.Richter@hogyros.de>
> 
> When using a Personal Access Token in Microsoft DevOps server, the username
> can be empty, so users might expect that pressing return on an username
> prompt will work.

I don't think this is a good idea.  libcurl relies on CURLOPT_USERPWD
being set to enable authentication, and before the appearance of
http.emptyAuth, it was extremely common for Kerberos users to specify an
empty username to get Git to authenticate properly.  I probably still
have some repositories on my system configured that way.

I believe GitHub can also accept an empty username with a PAT, but it
can also accept a dummy (e.g., "token"), which I would hope Azure DevOps
can do as well.  In such a case, the documentation for Azure DevOps
should just be updated to tell people to specify something like "token"
or their username.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]