Re: [PATCH 00/13] Support for arbitrary schemes in credentials

[M Hickford <mirth.hickford@gmail.com>]

On Sun, 24 Mar 2024 at 01:13, brian m. carlson
<sandals@crustytoothpaste.net> wrote:
>
> Right now, HTTP authentication in Git is mostly limited to approaches
> that require a username and password or are Kerberos (GSSAPI).  In
> addition, we effectively require that libcurl (or, for other software,
> such as Git LFS, using the credential helper, that HTTP library) knows
> how to implement the authentication scheme.
>
> However, this poses two sets of problems.  First, some sites, such as
> Azure DevOps, want to use Bearer authentication, which we don't support.

Git's lack of support for Bearer auth is also a frustration for OAuth
-- OAuth tokens are intended as Bearer tokens. As a workaround,
popular Git hosts support Basic auth with an OAuth token as password.
For certain hosts, this requires a magic username such as 'oauth2'
(GitLab) or 'x-token-auth' (Bitbucket). This is problematic if the
user has multiple accounts, or a different username stored in
credential.username or the remote URL.

> This is implemented using `http.extraHeader`, which is not a secure way
> to store credentials, since our credential helper protocol does not
> support this functionality.

Config key http.extraHeader is particularly unsuitable to hold
short-lived Bearer tokens. For example, GitLab's OAuth tokens expire
after 2 hours.

>
> In addition, other tools using the credential helper protocol do not
> support the variety of authentication mechanisms that Git does.
> Specifically, making NTLM function in a useful way on Windows is
> nontrivial and requires extensive integration and testing with C code,
> and because of this difficulty and the fact that NTLM uses cryptography
> known to be insecure since 1995, there is often little interest in
> implementing this support outside of libcurl. However, it would be
> helpful if people who want to use it can still use it.
>
> This series introduces new functionality to the credential helper
> protocol that allows helpers to produce credentials for arbitrary HTTP
> authentication schemes using the `authtype` and `credential`[0] fields.

Exciting! I drafted a patch to make git-credential-oauth generate
Bearer credentials
https://github.com/hickford/git-credential-oauth/pull/51 . Tested
together with your patch, it works for host bitbucket.org. Though of
course it isn't practical while storage helpers (such as
credential-cache, credential-wincred, credential-libsecret and
credential-osxkeychain) are unable to store the new attributes -- the
user has to complete the OAuth flow for every authentication.

> This allows a suitable credential helper to send Bearer credentials or
> any other standard or custom authentication scheme.  (It may be able to
> be extended to other functionality in the future, such as
> git-send-email, to implement custom SASL functionality, and due care has
> been taken to make the protocol adequately flexible for that purpose.)
>
> In addition, the protocol is also expanded to include per-helper state
> and multi-legged authentication (the former is effectively required for
> the latter).  The per-helper state can be useful to help credential
> helpers identify where the credential is stored, or any other
> information necessary.  Because NTLM and Negotiate (Kerberos/wrapped
> NTLM) require two rounds of authentication, the multi-legged
> authentication support along with per-helper state allows the helper to
> support these authentication methods without Git or other clients having
> to be aware of how they work.  (This would also be useful for SASL, as
> mentioned above.)
>
> This series introduces a capability mechanism to announce this
> functionality, which allows a helper to provide a username and password
> on older versions of Git while supporting more advanced functionality on
> newer versions.  (This is especially important on Azure DevOps, where
> NTLM uses a username and password but Basic or Bearer can use a personal
> access token.)  It is also designed such that extremely simple
> credential helpers, such as the shell one-liner in the Git FAQ that
> reads from the environment, don't accidentally claim to support
> functionality they don't offer.




>
> In addition, there is documentation for the expanded protocol, although
> none of the built-in helpers have been updated (that will be a future
> series for those for which it's possible).
>
> My personal interest here is getting credentials out of config files
> with `http.extraHeader` (which a future series will produce a warning
> for) and also allowing Git LFS to support Digest and NTLM with a
> suitable credential helper.  Git LFS used to support NTLM using custom
> code (because the Go standard library does not), but it was found to be
> broken in lots of ways on Windows, and nobody with a Windows system
> wanted to fix it or support it, so we removed it.  However, there are
> still some people who do want to use it, so allowing them to use a
> custom credential helper they maintain themselves seems like the best
> way forward.  Despite the advantages of this series for Azure DevOps, I
> have no personal or professional stake in their product; my only
> interest is the general one in whether their users can securely store
> credentials.  I believe the changes here are of general advantage to the
> Git userbase in a variety of ways such that the goal of this series
> should be uncontroversial.
>
> Feedback on any portion of this series is of course welcome.
>
> [0] A name different from `password` was explicitly chosen to avoid
> confusion from less capable protocol helpers so that they don't
> accidentally send invalid data.  This does have the downside that
> credential helpers must learn a new field to not log, but that should be
> generally easy to fix in most cases.
>
> brian m. carlson (13):
>   credential: add an authtype field
>   remote-curl: reset headers on new request
>   http: use new headers for each object request
>   credential: add a field for pre-encoded credentials
>   credential: gate new fields on capability
>   docs: indicate new credential protocol fields
>   http: add support for authtype and credential
>   credential: add an argument to keep state
>   credential: enable state capability
>   docs: set a limit on credential line length
>   t5563: refactor for multi-stage authentication
>   strvec: implement swapping two strvecs
>   credential: add support for multistage credential rounds
>
>  Documentation/git-credential.txt   |  59 +++++-
>  builtin/credential-cache--daemon.c |   2 +-
>  builtin/credential-store.c         |   2 +-
>  builtin/credential.c               |   7 +-
>  credential.c                       | 114 ++++++++++-
>  credential.h                       |  69 ++++++-
>  http.c                             | 128 +++++++-----
>  http.h                             |   5 +
>  imap-send.c                        |   2 +-
>  remote-curl.c                      |  14 +-
>  strvec.c                           |   7 +
>  strvec.h                           |   5 +
>  t/lib-httpd/nph-custom-auth.sh     |  17 +-
>  t/t0300-credentials.sh             | 136 ++++++++++++-
>  t/t5563-simple-http-auth.sh        | 308 +++++++++++++++++++++++++----
>  15 files changed, 760 insertions(+), 115 deletions(-)
>