OAuth2 support in git?

OAuth2 support in git?

[Christian Halstrick]

Can I use native git as client to contact a git server which does
authentication with OAuth2 Client Credentials Grant [1]?

Background: We are running gerrit based git servers [2] in a cloud
environment. That environment supports OAuth2 authorization for the
apps running in the cloud. The idea is that clients (e.g. jenkins
jobs) talking git over http with such git servers should be able to
use OAuth2 tokens to authenticate clone/fetch requests. We would have
to adapt gerrit source code for token handling/validation but I am
asking here about the client side.

I know that other git server environments like github support that on
client side by allowing tokens to be used as usernames in a BASIC
authentication flow. We could do the same but I am asking whether
there is also a way to transport tokens in a standard conform
"Authorization: Bearer ..." Header field.

[1] https://tools.ietf.org/html/rfc6749#section-4.4
[2] https://www.gerritcodereview.com/

Re: OAuth2 support in git?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1330 bytes --]

On Thu, Jun 14, 2018 at 10:09:39AM +0200, Christian Halstrick wrote:
> Can I use native git as client to contact a git server which does
> authentication with OAuth2 Client Credentials Grant [1]?
> 
> Background: We are running gerrit based git servers [2] in a cloud
> environment. That environment supports OAuth2 authorization for the
> apps running in the cloud. The idea is that clients (e.g. jenkins
> jobs) talking git over http with such git servers should be able to
> use OAuth2 tokens to authenticate clone/fetch requests. We would have
> to adapt gerrit source code for token handling/validation but I am
> asking here about the client side.
> 
> I know that other git server environments like github support that on
> client side by allowing tokens to be used as usernames in a BASIC
> authentication flow. We could do the same but I am asking whether
> there is also a way to transport tokens in a standard conform
> "Authorization: Bearer ..." Header field.

There isn't any support for Bearer authentication in Git.  For HTTP, we
use libcurl, which doesn't provide this natively.  While it could in
theory be added, it would require some reworking of the auth code.

You are, of course, welcome to send a patch.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]

Re: OAuth2 support in git?

[Jeff King]

On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:

> > I know that other git server environments like github support that on
> > client side by allowing tokens to be used as usernames in a BASIC
> > authentication flow. We could do the same but I am asking whether
> > there is also a way to transport tokens in a standard conform
> > "Authorization: Bearer ..." Header field.
> 
> There isn't any support for Bearer authentication in Git.  For HTTP, we
> use libcurl, which doesn't provide this natively.  While it could in
> theory be added, it would require some reworking of the auth code.
> 
> You are, of course, welcome to send a patch.

If it's just a custom Authorization header, we should be able to support
it with existing curl versions without _too_ much effort.

I think there are probably two possible directions:

 1. add a special "bearer" command line option, etc, as a string

 2. add a boolean option to send the existing "password" field as a
    "bearer" header

I suspect (2) would fit in with the existing code better, as the special
case would mostly be limited to the manner in which we feed the
credential to curl. And you could probably just set a config option for
"this url's auth will be oauth2", and use the existing mechanisms for
providing the password.

We'd maybe also want to allow credential helpers to say "by the way,
this password should be treated as a bearer token", for cases where you
might sometimes use oauth2 and sometimes a real password.

-Peff

RE: OAuth2 support in git?

[Randall S. Becker]

On June 14, 2018 11:15 AM, Jeff King wrote:
> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> 
> > > I know that other git server environments like github support that
> > > on client side by allowing tokens to be used as usernames in a BASIC
> > > authentication flow. We could do the same but I am asking whether
> > > there is also a way to transport tokens in a standard conform
> > > "Authorization: Bearer ..." Header field.
> >
> > There isn't any support for Bearer authentication in Git.  For HTTP,
> > we use libcurl, which doesn't provide this natively.  While it could
> > in theory be added, it would require some reworking of the auth code.
> >
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support it
> with existing curl versions without _too_ much effort.
> 
> I think there are probably two possible directions:
> 
>  1. add a special "bearer" command line option, etc, as a string
> 
>  2. add a boolean option to send the existing "password" field as a
>     "bearer" header
> 
> I suspect (2) would fit in with the existing code better, as the special case
> would mostly be limited to the manner in which we feed the credential to
> curl. And you could probably just set a config option for "this url's auth will
> be oauth2", and use the existing mechanisms for providing the password.
> 
> We'd maybe also want to allow credential helpers to say "by the way, this
> password should be treated as a bearer token", for cases where you might
> sometimes use oauth2 and sometimes a real password.

Be aware that there are 4 (ish) flavours of OAuth2 the last time I checked. It is important to know which one (or all) to implement. The embedded form is probably the easiest to comprehend - and the least implemented from my research. More common OAuth2 instances use a third-man website to hold session keys and authorization. That may be problematic for a whole bunch of us who do not play in that world.

Cheers,
Randall

-- Brief whoami:
  NonStop developer since approximately NonStop(211288444200000000)
  UNIX developer since approximately 421664400
-- In my real life, I talk too much.

Re: OAuth2 support in git?

[Jeff King]

On Thu, Jun 14, 2018 at 04:46:10PM -0400, Randall S. Becker wrote:

> > I suspect (2) would fit in with the existing code better, as the special case
> > would mostly be limited to the manner in which we feed the credential to
> > curl. And you could probably just set a config option for "this url's auth will
> > be oauth2", and use the existing mechanisms for providing the password.
> > 
> > We'd maybe also want to allow credential helpers to say "by the way, this
> > password should be treated as a bearer token", for cases where you might
> > sometimes use oauth2 and sometimes a real password.
> 
> Be aware that there are 4 (ish) flavours of OAuth2 the last time I
> checked. It is important to know which one (or all) to implement. The
> embedded form is probably the easiest to comprehend - and the least
> implemented from my research. More common OAuth2 instances use a
> third-man website to hold session keys and authorization. That may be
> problematic for a whole bunch of us who do not play in that world.

I think Git's usage would be limited to "how do I present this token for
my requests". I don't think we'd ever recognize an oauth redirect and
try to fulfill it ourselves.  We'd rely on getting a 401 and punting all
those bits to a credential helper to do the heavy lifting.

I say that not knowing much about oauth2, of course, so maybe there
would be complications with that approach (I do know there are multiple
ways you can present a token, but we'd support whichever ones people are
interested in enough to show up and provide a patch for).

-Peff

Re: OAuth2 support in git?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

On Thu, Jun 14, 2018 at 11:15:07AM -0400, Jeff King wrote:
> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> > There isn't any support for Bearer authentication in Git.  For HTTP, we
> > use libcurl, which doesn't provide this natively.  While it could in
> > theory be added, it would require some reworking of the auth code.
> > 
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support
> it with existing curl versions without _too_ much effort.

It shouldn't be too difficult, but we have some fallback among various
authentication types that would need reworking.

> I think there are probably two possible directions:
> 
>  1. add a special "bearer" command line option, etc, as a string
> 
>  2. add a boolean option to send the existing "password" field as a
>     "bearer" header
> 
> I suspect (2) would fit in with the existing code better, as the special
> case would mostly be limited to the manner in which we feed the
> credential to curl. And you could probably just set a config option for
> "this url's auth will be oauth2", and use the existing mechanisms for
> providing the password.

I agree option (2) would be better.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]

Re: OAuth2 support in git?

[Johannes Schindelin]

Hi Peff,

On Thu, 14 Jun 2018, Jeff King wrote:

> On Thu, Jun 14, 2018 at 10:13:42AM +0000, brian m. carlson wrote:
> 
> > > I know that other git server environments like github support that on
> > > client side by allowing tokens to be used as usernames in a BASIC
> > > authentication flow. We could do the same but I am asking whether
> > > there is also a way to transport tokens in a standard conform
> > > "Authorization: Bearer ..." Header field.
> > 
> > There isn't any support for Bearer authentication in Git.  For HTTP, we
> > use libcurl, which doesn't provide this natively.  While it could in
> > theory be added, it would require some reworking of the auth code.
> > 
> > You are, of course, welcome to send a patch.
> 
> If it's just a custom Authorization header, we should be able to support
> it with existing curl versions without _too_ much effort.

Indeed. Because it is already implemented:

	git -c http.extraheader="Authorization: Bearer ..." ...

To make this a *little* safer, you can use http.<URL>.extraheader.

Ciao,
Dscho

Re: OAuth2 support in git?

[Jeff King]

On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:

> > If it's just a custom Authorization header, we should be able to support
> > it with existing curl versions without _too_ much effort.
> 
> Indeed. Because it is already implemented:
> 
> 	git -c http.extraheader="Authorization: Bearer ..." ...
> 
> To make this a *little* safer, you can use http.<URL>.extraheader.

Yeah, that will work for some cases. A few places it might not:

 - some people may want to provide this only in response to a 401

 - some tokens may need to be refreshed, which would require interacting
   with a credential helper to do the rest of the oauth conversation

 - there's no good way to hide your token in secure storage (versus
   sticking it on the command-line or in a config file).

-Peff

Re: OAuth2 support in git?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Sun, Jun 17, 2018 at 01:37:24PM +0200, Johannes Schindelin wrote:
>
>> > If it's just a custom Authorization header, we should be able to support
>> > it with existing curl versions without _too_ much effort.
>> 
>> Indeed. Because it is already implemented:
>> 
>> 	git -c http.extraheader="Authorization: Bearer ..." ...
>> 
>> To make this a *little* safer, you can use http.<URL>.extraheader.
>
> Yeah, that will work for some cases. A few places it might not:
>
>  - some people may want to provide this only in response to a 401
>
>  - some tokens may need to be refreshed, which would require interacting
>    with a credential helper to do the rest of the oauth conversation
>
>  - there's no good way to hide your token in secure storage (versus
>    sticking it on the command-line or in a config file).

And all of these three are what you get for free by building on the
credential helper framework, after extending it a bit so that the
filled credential structure can tell the http code to show it to the
other side as a bearer token, not a password or password hash.  The
helper is asked to supply the auth material only after 401, which
covers both the first and the second points, and then keeping the
auth material in-core (e.g. cache--daemon) would be more secure
which covers the third point.  Am I following you correctly?

Thanks.

Re: OAuth2 support in git?

[Jeff King]

On Mon, Jun 18, 2018 at 08:53:27AM -0700, Junio C Hamano wrote:

> > Yeah, that will work for some cases. A few places it might not:
> >
> >  - some people may want to provide this only in response to a 401
> >
> >  - some tokens may need to be refreshed, which would require interacting
> >    with a credential helper to do the rest of the oauth conversation
> >
> >  - there's no good way to hide your token in secure storage (versus
> >    sticking it on the command-line or in a config file).
> 
> And all of these three are what you get for free by building on the
> credential helper framework, after extending it a bit so that the
> filled credential structure can tell the http code to show it to the
> other side as a bearer token, not a password or password hash.  The
> helper is asked to supply the auth material only after 401, which
> covers both the first and the second points, and then keeping the
> auth material in-core (e.g. cache--daemon) would be more secure
> which covers the third point.  Am I following you correctly?

Yes, exactly.

Even if the credential protocol itself doesn't learn about this feature,
even a config option for "treat password as token to send via bearer"
would help. The "how" of sending the token isn't secret, just the token
itself. So everything else can just pretend it's a password (it's a
little funny because I think there isn't a matching username, but you
could probably get by with an empty one).

That's all just off the top of my head without digging back into the
code, nor running any experiments, of course. There may be some gotchas. :)

-Peff

Re: OAuth2 support in git?

[Christian Halstrick]

What is not clear to me is how we can make use of the servers initial
response in
order control which credential helper to call and how to transport the
credentials.

Imagine we try to clone over http. The initial request sent to the server
may not contain a "Authorization: ..." header and the server responds
with Unauthorized.
But the server response contains hints like a "WWW-Authenticate: Basic
realm=..." line
or a "WWW-Authenticate: Bearer realm=..." line which helps choosing the
authentication scheme used next. Maybe the server even responds with both lines
telling I would accept BASIC or BEARER.

I can imagine that we want libcurl to deal with that decisions. But
even then. How
do we make sure the our credential helpers can act return either user/password
or bearer tokens based on the server response? If credential helper
would have access
to the servers response (or only relevant parts of it?) it could
decide whether to
feel responsible for that server or not and what data to return.

And if credential helper could optionally give metadata about the kind
credential they offer
(e.g. "I return user/password" or "I return a bearer token") then core
code could know
where to transport this data. E.g. in a "Authorization: Basic ..." or
a "Authorization: Bearer ..."
field.

Ciao
  Chris

Re: OAuth2 support in git?

[Jeff King]

On Tue, Jun 19, 2018 at 02:36:50PM +0200, Christian Halstrick wrote:

> What is not clear to me is how we can make use of the servers initial
> response in order control which credential helper to call and how to
> transport the credentials.

I don't think we'd ever decide _which_ credential helper to call; we
always call all of them, in order, and then quit when we have sufficient
credentials to continue.

But potentially we could feed some extra information to each helper and
let it decide what to do.

> Imagine we try to clone over http. The initial request sent to the
> server may not contain a "Authorization: ..." header and the server
> responds with Unauthorized.  But the server response contains hints
> like a "WWW-Authenticate: Basic realm=..." line or a
> "WWW-Authenticate: Bearer realm=..." line which helps choosing the
> authentication scheme used next. Maybe the server even responds with
> both lines telling I would accept BASIC or BEARER.

So for this example, yeah, I think it might make sense to feed the
credential helper extra context like "authtype=basic" or similar. Most
helpers would ignore it, but smart ones could make a decision based on
it.

And then the response could contain a similar "authtype" key in the
response.

> I can imagine that we want libcurl to deal with that decisions. But
> even then. How do we make sure the our credential helpers can act
> return either user/password or bearer tokens based on the server
> response? If credential helper would have access to the servers
> response (or only relevant parts of it?) it could decide whether to
> feel responsible for that server or not and what data to return.
> 
> And if credential helper could optionally give metadata about the kind
> credential they offer (e.g. "I return user/password" or "I return a
> bearer token") then core code could know where to transport this data.
> E.g. in a "Authorization: Basic ..." or a "Authorization: Bearer ..."
> field.

Yep, I think that all matches my general line of thinking. It would help
if we had some concrete cases. In particular, it's unclear to me if:

  1. A config option to say "treat password as a bearer token" would be
     enough.

  2. We'd need the credential helper to say "I'm giving you a token"
     versus "I'm giving you a password".

  3. We might need _both_ (1) and (2), because some servers would be
     fine with (1) and it lets them Just Work with credential helpers
     that are unaware of bearer tokens in the first place.

I suspect the answer is (3), but I'd probably delay working on (2) until
I saw a situation that really needed it. :)

But I think we're on the same page, so if you're looking into or
developing more concrete cases, those answers should become more clear.

-Peff