[PATCH] credential: add nocache option to the credentials API

[PATCH] credential: add nocache option to the credentials API

[Masaya Suzuki]

The credentials API calls credentials helpers in order. If a
username/password pair is returned the helpers and if it's used for
authentication successfully, it's announced to the helpers and they can
store it for later use.

Some credentials are valid only for the limited time and should not be
cached. In this case, because the credential is announced to all helpers
and they can independently decide whether they will cache it or not,
those short-lived credentials can be cached.

This change adds an option that a credential helper can specify that the
credential returned by the helper should not be cached. If this is
specified, even after the credential is used successfully, it won't be
announced to other helpers for store.

Signed-off-by: Masaya Suzuki <masayasuzuki@google.com>
---
 Documentation/technical/api-credentials.txt | 4 +++-
 credential.c                                | 4 +++-
 credential.h                                | 3 ++-
 t/t0300-credentials.sh                      | 9 +++++++++
 4 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/Documentation/technical/api-credentials.txt b/Documentation/technical/api-credentials.txt
index 75368f26ca..3db5841b40 100644
--- a/Documentation/technical/api-credentials.txt
+++ b/Documentation/technical/api-credentials.txt
@@ -251,7 +251,9 @@ even no values at all if it has nothing useful to provide. Any provided
 attributes will overwrite those already known about by Git.  If a helper
 outputs a `quit` attribute with a value of `true` or `1`, no further
 helpers will be consulted, nor will the user be prompted (if no
-credential has been provided, the operation will then fail).
+credential has been provided, the operation will then fail). If a helper outputs
+a `nocache` attribute with a value of `true` or `1`, `credential_approve` will
+not be called even after the credential is used for authentication sucessfully.
 
 For a `store` or `erase` operation, the helper's output is ignored.
 If it fails to perform the requested operation, it may complain to
diff --git a/credential.c b/credential.c
index 62be651b03..db7b351447 100644
--- a/credential.c
+++ b/credential.c
@@ -179,6 +179,8 @@ int credential_read(struct credential *c, FILE *fp)
 			credential_from_url(c, value);
 		} else if (!strcmp(key, "quit")) {
 			c->quit = !!git_config_bool("quit", value);
+		} else if (!strcmp(key, "nocache")) {
+			c->no_cache= !!git_config_bool("nocache", value);
 		}
 		/*
 		 * Ignore other lines; we don't know what they mean, but
@@ -296,7 +298,7 @@ void credential_approve(struct credential *c)
 {
 	int i;
 
-	if (c->approved)
+	if (c->approved || c->no_cache)
 		return;
 	if (!c->username || !c->password)
 		return;
diff --git a/credential.h b/credential.h
index 6b0cd16be2..be0f35d841 100644
--- a/credential.h
+++ b/credential.h
@@ -8,7 +8,8 @@ struct credential {
 	unsigned approved:1,
 		 configured:1,
 		 quit:1,
-		 use_http_path:1;
+		 use_http_path:1,
+		 no_cache:1;
 
 	char *username;
 	char *password;
diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh
index 82eaaea0f4..ad06f6fe11 100755
--- a/t/t0300-credentials.sh
+++ b/t/t0300-credentials.sh
@@ -118,6 +118,15 @@ test_expect_success 'do not bother storing password-less credential' '
 	EOF
 '
 
+test_expect_success 'credential_approve does not call helpers for nocache' '
+	check approve useless <<-\EOF
+	username=foo
+	password=bar
+	nocache=1
+	--
+	--
+	EOF
+'
 
 test_expect_success 'credential_reject calls all helpers' '
 	check reject useless "verbatim one two" <<-\EOF
-- 
2.22.0.410.gd8fdbe21b5-goog

Re: [PATCH] credential: add nocache option to the credentials API

[Jeff King]

On Sat, Jul 06, 2019 at 10:51:32PM -0700, Masaya Suzuki wrote:

> The credentials API calls credentials helpers in order. If a
> username/password pair is returned the helpers and if it's used for
> authentication successfully, it's announced to the helpers and they can
> store it for later use.
> 
> Some credentials are valid only for the limited time and should not be
> cached. In this case, because the credential is announced to all helpers
> and they can independently decide whether they will cache it or not,
> those short-lived credentials can be cached.
> 
> This change adds an option that a credential helper can specify that the
> credential returned by the helper should not be cached. If this is
> specified, even after the credential is used successfully, it won't be
> announced to other helpers for store.

I think this makes sense to do, though note that there's an old
discussion which covers some alternatives:

  https://public-inbox.org/git/20120407033417.GA13914@sigill.intra.peff.net/

In that patch, I essentially proposed making all gathered credentials as
nocache. That's a more secure default (though in some cases less
convenient).

It did break a case Shawn had of caching the result of another helper. I
showed some options there for providing a mechanism to chain helpers
together explicitly.

We also discussed helpers passing out an explicit ttl. That's a more
general case of your nocache flag (i.e., ttl=0 covers that case, but we
could additionally pass "ttl" to the cache helper to let it be smarter).

Given the age of that discussion and the fact that nobody has really
complained much in the interim, I'm OK to go with your much simpler
approach. But I think it's worth at least thinking for a few minutes on
whether there's anything to pull from that discussion. :)

(As a side note, I've had all those patches on my "to revisit and send
upstream" queue for 7 years; if we take yours, maybe I can finally let
them go. ;) ).

>  Documentation/technical/api-credentials.txt | 4 +++-
>  credential.c                                | 4 +++-
>  credential.h                                | 3 ++-
>  t/t0300-credentials.sh                      | 9 +++++++++
>  4 files changed, 17 insertions(+), 3 deletions(-)

The patch itself looks good; two minor comments:

> @@ -296,7 +298,7 @@ void credential_approve(struct credential *c)
>  {
>  	int i;
>  
> -	if (c->approved)
> +	if (c->approved || c->no_cache)
>  		return;
>  	if (!c->username || !c->password)
>  		return;

Here we're disallowing a "nocache" credential from being passed to _any_
helper, whether it's caching or not. It could be storing permanently,
though perhaps that's semantic nitpicking (if it's not to be cached, it
probably shouldn't be stored permanently either). Other helpers could in
theory be doing something else with the data, though in practice I doubt
here are any uses beyond debugging.

I dunno. I started writing that paragraph to suggest calling it
"nostore" or something, but I think that is really no better than
"nocache".

If we did have a ttl, I'd expect to see a check for "ttl=0" here, but
then otherwise pass the ttl along to the helper (and a matching change
in credential-cache to use the minimum of the credential-specific ttl or
the global-configured one).

> diff --git a/t/t0300-credentials.sh b/t/t0300-credentials.sh
> index 82eaaea0f4..ad06f6fe11 100755
> --- a/t/t0300-credentials.sh
> +++ b/t/t0300-credentials.sh
> @@ -118,6 +118,15 @@ test_expect_success 'do not bother storing password-less credential' '
>  	EOF
>  '
>  
> +test_expect_success 'credential_approve does not call helpers for nocache' '
> +	check approve useless <<-\EOF
> +	username=foo
> +	password=bar
> +	nocache=1
> +	--
> +	--
> +	EOF
> +'

At first I thought this test was doing nothing, since we don't generally
expect a helper to produce output for an "approve" request (and there
are cases in lib-credential.sh that rely on that). But the "useless"
helper is intentionally chatty, so it produces output for all requests,
and this would fail without your patch. Good.

The cases in lib-credential also omit the "--" for empty sections, but I
see the similar "do not bother..." test above includes them. It
shouldn't matter either way.

So I think this looks good.

-Peff

Re: [PATCH] credential: add nocache option to the credentials API

[Masaya Suzuki]

On Tue, Jul 9, 2019 at 5:56 AM Jeff King <peff@peff.net> wrote:
>
> On Sat, Jul 06, 2019 at 10:51:32PM -0700, Masaya Suzuki wrote:
>
> > The credentials API calls credentials helpers in order. If a
> > username/password pair is returned the helpers and if it's used for
> > authentication successfully, it's announced to the helpers and they can
> > store it for later use.
> >
> > Some credentials are valid only for the limited time and should not be
> > cached. In this case, because the credential is announced to all helpers
> > and they can independently decide whether they will cache it or not,
> > those short-lived credentials can be cached.
> >
> > This change adds an option that a credential helper can specify that the
> > credential returned by the helper should not be cached. If this is
> > specified, even after the credential is used successfully, it won't be
> > announced to other helpers for store.
>
> I think this makes sense to do, though note that there's an old
> discussion which covers some alternatives:
>
>   https://public-inbox.org/git/20120407033417.GA13914@sigill.intra.peff.net/
>
> In that patch, I essentially proposed making all gathered credentials as
> nocache. That's a more secure default (though in some cases less
> convenient).
>
> It did break a case Shawn had of caching the result of another helper. I
> showed some options there for providing a mechanism to chain helpers
> together explicitly.

I think that it's better to make it nocache by default. Having one
helper produce a credential and having another cache it looks storage.
But since this is the current behavior, I'm OK with just keeping
nocache an option. It's backward compatible.

> We also discussed helpers passing out an explicit ttl. That's a more
> general case of your nocache flag (i.e., ttl=0 covers that case, but we
> could additionally pass "ttl" to the cache helper to let it be smarter).

TTL sounds like it's a generalized version. It might be a bit awkward
because the existing credential helpers that don't support TTL would
anyway cache the credentials. I think in practice the password saving
feature is mainly used by those password management software (like
git-credential-osxkeychain), and they wouldn't support a short-lived
credential. Just having nocache seems fine to me. As you said, if
needed, "ttl" can be added and "nocache" can be just a shorthand of
"ttl=0".

> Given the age of that discussion and the fact that nobody has really
> complained much in the interim, I'm OK to go with your much simpler
> approach. But I think it's worth at least thinking for a few minutes on
> whether there's anything to pull from that discussion. :)
>
> (As a side note, I've had all those patches on my "to revisit and send
> upstream" queue for 7 years; if we take yours, maybe I can finally let
> them go. ;) ).
>
> >  Documentation/technical/api-credentials.txt | 4 +++-
> >  credential.c                                | 4 +++-
> >  credential.h                                | 3 ++-
> >  t/t0300-credentials.sh                      | 9 +++++++++
> >  4 files changed, 17 insertions(+), 3 deletions(-)
>
> The patch itself looks good; two minor comments:
>
> > @@ -296,7 +298,7 @@ void credential_approve(struct credential *c)
> >  {
> >       int i;
> >
> > -     if (c->approved)
> > +     if (c->approved || c->no_cache)
> >               return;
> >       if (!c->username || !c->password)
> >               return;
>
> Here we're disallowing a "nocache" credential from being passed to _any_
> helper, whether it's caching or not. It could be storing permanently,
> though perhaps that's semantic nitpicking (if it's not to be cached, it
> probably shouldn't be stored permanently either). Other helpers could in
> theory be doing something else with the data, though in practice I doubt
> here are any uses beyond debugging.

I cannot think of a usage either. If there's a good usage, I would
change this, but if it's for debugging, it's better to be done with
those debugging features (like GIT_TRACE_CURL). Note that this is
called only when the credential is successfully used. We probably want
to use such debugging feature for the credentials that are not
successfully used.

Re: [PATCH] credential: add nocache option to the credentials API

[Jeff King]

On Mon, Jul 22, 2019 at 10:30:52AM -0700, Masaya Suzuki wrote:

> > In that patch, I essentially proposed making all gathered credentials as
> > nocache. That's a more secure default (though in some cases less
> > convenient).
> >
> > It did break a case Shawn had of caching the result of another helper. I
> > showed some options there for providing a mechanism to chain helpers
> > together explicitly.
> 
> I think that it's better to make it nocache by default. Having one
> helper produce a credential and having another cache it looks storage.
> But since this is the current behavior, I'm OK with just keeping
> nocache an option. It's backward compatible.

Yeah, I think that make sense.

> > We also discussed helpers passing out an explicit ttl. That's a more
> > general case of your nocache flag (i.e., ttl=0 covers that case, but we
> > could additionally pass "ttl" to the cache helper to let it be smarter).
> 
> TTL sounds like it's a generalized version. It might be a bit awkward
> because the existing credential helpers that don't support TTL would
> anyway cache the credentials. I think in practice the password saving
> feature is mainly used by those password management software (like
> git-credential-osxkeychain), and they wouldn't support a short-lived
> credential. Just having nocache seems fine to me. As you said, if
> needed, "ttl" can be added and "nocache" can be just a shorthand of
> "ttl=0".

I was thinking that Git itself could treat "ttl=0" specially, the same
as your nocache, and avoid passing it along to any helpers during the
approve stage. That would make it exactly equivalent to your patch
(modulo the name change).

> > Here we're disallowing a "nocache" credential from being passed to _any_
> > helper, whether it's caching or not. It could be storing permanently,
> > though perhaps that's semantic nitpicking (if it's not to be cached, it
> > probably shouldn't be stored permanently either). Other helpers could in
> > theory be doing something else with the data, though in practice I doubt
> > here are any uses beyond debugging.
> 
> I cannot think of a usage either. If there's a good usage, I would
> change this, but if it's for debugging, it's better to be done with
> those debugging features (like GIT_TRACE_CURL). Note that this is
> called only when the credential is successfully used. We probably want
> to use such debugging feature for the credentials that are not
> successfully used.

Yeah, I don't think debugging is worth caring about here. As you say, we
can dump the data readily through other means. I was more wondering if
there was some legitimate use where a helper wanted to see (but not
store!) an existing credential. But again, I don't know of one.

And as you noted above, if we don't suppress the helper calls inside
Git, then every matching storage helper needs to learn about "nocache"
(or "ttl") before it will do any good.

-Peff

Re: [PATCH] credential: add nocache option to the credentials API

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I was thinking that Git itself could treat "ttl=0" specially, the same
> as your nocache, and avoid passing it along to any helpers during the
> approve stage. That would make it exactly equivalent to your patch
> (modulo the name change).
> ...
> And as you noted above, if we don't suppress the helper calls inside
> Git, then every matching storage helper needs to learn about "nocache"
> (or "ttl") before it will do any good.

I was waiting for this discussion to settle and then the discussion
seems to have petered out.  Any interest to following the "ttl with
special casing value 0 as 'nocache'" idea thru from either two of
you, or should I take the patch as is in the meantime?

Thanks.

Re: [PATCH] credential: add nocache option to the credentials API

[Masaya Suzuki]

On Mon, Aug 26, 2019 at 9:28 AM Junio C Hamano <gitster@pobox.com> wrote:
>
> Jeff King <peff@peff.net> writes:
>
> > I was thinking that Git itself could treat "ttl=0" specially, the same
> > as your nocache, and avoid passing it along to any helpers during the
> > approve stage. That would make it exactly equivalent to your patch
> > (modulo the name change).
> > ...
> > And as you noted above, if we don't suppress the helper calls inside
> > Git, then every matching storage helper needs to learn about "nocache"
> > (or "ttl") before it will do any good.
>
> I was waiting for this discussion to settle and then the discussion
> seems to have petered out.  Any interest to following the "ttl with
> special casing value 0 as 'nocache'" idea thru from either two of
> you, or should I take the patch as is in the meantime?

Sorry for the late reply. I think about this again. I imagine that, if
I would like to have credentials with an expiration and I want to have
them managed by other helpers, it's probably better to use an absolute
timestamp instead of duration. The second call to the helpers is done
after the remote call. For the helpers that store TTL-ed credentials,
they cannot tell the start time of the TTL in the second call. This
makes it hard to cache the short-lived credentials safely because some
time has spent during the remote call and the actual TTL is shorter
than ttl=N option. From this, instead of adding ttl=DURATION, it might
be better to have expires_at=TIMESTAMP.

Maybe my observation is not an issue. I don't know. For now, adding
nocache seems a safer action for me, so I vote for taking nocache
patch as-is in the meantime. If there's somebody who wants to receive
TTL or expiration timestamp, they can decide what's actually needed
later.