[PATCH] docs: clarify that credential discards unrecognised attributes

[PATCH] docs: clarify that credential discards unrecognised attributes

[M Hickford via GitGitGadget]

From: M Hickford <mirth.hickford@gmail.com>

It was previously unclear how unrecognised attributes are handled.

Signed-off-by: M Hickford <mirth.hickford@gmail.com>
---
    docs: clarify that credential discards unrecognised attributes
    
    It was previously unclear how unrecognised attributes are handled.
    
    Signed-off-by: M Hickford mirth.hickford@gmail.com

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1393%2Fhickford%2Funrecognised-attributes-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1393/hickford/unrecognised-attributes-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1393

 Documentation/git-credential.txt | 2 ++
 Documentation/gitcredentials.txt | 1 +
 2 files changed, 3 insertions(+)

diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt
index f18673017f5..ac2818b9f66 100644
--- a/Documentation/git-credential.txt
+++ b/Documentation/git-credential.txt
@@ -160,6 +160,8 @@ empty string.
 Components which are missing from the URL (e.g., there is no
 username in the example above) will be left unset.
 
+Unrecognised attributes are silently discarded.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/Documentation/gitcredentials.txt b/Documentation/gitcredentials.txt
index 80517b4eb2c..e856e3c8330 100644
--- a/Documentation/gitcredentials.txt
+++ b/Documentation/gitcredentials.txt
@@ -269,6 +269,7 @@ stdout in the same format (see linkgit:git-credential[1] for common
 attributes). A helper is free to produce a subset, or even no values at
 all if it has nothing useful to provide. Any provided attributes will
 overwrite those already known about by Git's credential subsystem.
+Unrecognised attributes are silently discarded.
 
 While it is possible to override all attributes, well behaving helpers
 should refrain from doing so for any attribute other than username and

base-commit: 1fc3c0ad407008c2f71dd9ae1241d8b75f8ef886
-- 
gitgitgadget

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[Jeff King]

On Mon, Oct 24, 2022 at 07:57:48AM +0000, M Hickford via GitGitGadget wrote:

> It was previously unclear how unrecognised attributes are handled.

Yeah, this was always part of the intended behavior, but I agree we did
not say it very explicitly (aside from an in-code comment!). Both the
intent and content of your patch look good to me.

We did discuss patches a long time ago that would let Git carry
arbitrary keys between helpers, even if Git itself didn't understand it.
One of the intended uses was to let helpers talk to each other about
TTLs. So if you had say:

  [credential]
  helper = generate-some-token
  helper = cache

where the first helper generates a token, and the second caches it, the
first one could shove a "ttl" or "expiration" key into the protocol,
which the cache could then learn to respect.

But we never merged such a thing, and in practice I think people would
just implement both parts as a single helper for simplicity. And anyway,
even if we did want to do that, such a patch would want to modify the
documentation to explain the new behavior. :)

-Peff

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[Jeff King]

On Mon, Oct 24, 2022 at 07:59:22PM -0400, Jeff King wrote:

> We did discuss patches a long time ago that would let Git carry
> arbitrary keys between helpers, even if Git itself didn't understand it.
> One of the intended uses was to let helpers talk to each other about
> TTLs. So if you had say:
> 
>   [credential]
>   helper = generate-some-token
>   helper = cache
> 
> where the first helper generates a token, and the second caches it, the
> first one could shove a "ttl" or "expiration" key into the protocol,
> which the cache could then learn to respect.

In case anyone is morbidly curious, it was in this thread:

  https://lore.kernel.org/git/20120407033417.GA13914@sigill.intra.peff.net/

-Peff

Thanks

[M Hickford]

Thanks Jeff for your reply. This is helpful to understand the background.

(first message using git send-email, hopefully I followed the instructions correctly)

Re: Thanks

[Bagas Sanjaya]

On 10/25/22 08:51, M Hickford wrote:
> Thanks Jeff for your reply. This is helpful to understand the background.
> 
> (first message using git send-email, hopefully I followed the instructions correctly)

You messed up the thread (you broke it).

On every message showed on lore.kernel.org/git, there is reply
instructions at the bottom of the message's page. Follow it closely.

If you insisted on either using git-send-email(1) or mailto: link
to reply, you need to manually quote any portion of original message
you wish to give the context. Make sure every line of quoted text
starts with `> `.

Better yet, reply using "reply-all" button or key in your mail client.
However, make sure your email is text/plain (not HTML) and your mail
client doesn't mess up with whitespaces (convert tabs to spaces,
word wrapping).

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

Re: Thanks

[M Hickford]

> > (first message using git send-email, hopefully I followed the instructions correctly)
>
> You messed up the thread (you broke it).

Curious. The thread overview "5+ messages" at https://lore.kernel.org/git/20221025015116.4730-1-mirth.hickford@gmail.com/#related looks okay to me. I followed the git send-email instructions setting the In-Reply-To header. I changed the subject -- maybe that confused some clients? Thanks for the tips.

Re: Thanks

[Jeff King]

On Wed, Oct 26, 2022 at 05:39:05AM +0100, M Hickford wrote:

> > > (first message using git send-email, hopefully I followed the instructions correctly)
> >
> > You messed up the thread (you broke it).
> 
> Curious. The thread overview "5+ messages" at
> https://lore.kernel.org/git/20221025015116.4730-1-mirth.hickford@gmail.com/#related
> looks okay to me. I followed the git send-email instructions setting
> the In-Reply-To header. I changed the subject -- maybe that confused
> some clients?

Your reply looked fine to me. The in-reply-to you used is correct. It's
OK to change the subject (though usually only done when the conversation
drifts to a different topic).

-Peff

Re: Thanks

[Junio C Hamano]

M Hickford <mirth.hickford@gmail.com> writes:

>> > (first message using git send-email, hopefully I followed the instructions correctly)
>>
>> You messed up the thread (you broke it).
>
> Curious. The thread overview "5+ messages" at
> https://lore.kernel.org/git/20221025015116.4730-1-mirth.hickford@gmail.com/#related
> looks okay to me. I followed the git send-email instructions
> setting the In-Reply-To header. I changed the subject -- maybe
> that confused some clients? Thanks for the tips.

Your In-Reply-To: header looked OK to me.  Some webmail platforms
break the thread when Subject: is edited, but that is their bug.  As
long as you keep the In-Reply-To: chain correct, retitling the
message as necessary is indeed encouraged practice.

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[M Hickford]

On Tue, 25 Oct 2022 at 00:59, Jeff King <peff@peff.net> wrote:
>
> On Mon, Oct 24, 2022 at 07:57:48AM +0000, M Hickford via GitGitGadget wrote:
>
> > It was previously unclear how unrecognised attributes are handled.
>
> Yeah, this was always part of the intended behavior, but I agree we did
> not say it very explicitly (aside from an in-code comment!). Both the
> intent and content of your patch look good to me.

Thanks. What happens next? I should look for this change in the seen
branch? https://git-scm.com/docs/MyFirstContribution#after-approval

> We did discuss patches a long time ago that would let Git carry
> arbitrary keys between helpers, even if Git itself didn't understand it.
> One of the intended uses was to let helpers talk to each other about
> TTLs. So if you had say:
>
>   [credential]
>   helper = generate-some-token
>   helper = cache
>
> where the first helper generates a token, and the second caches it, the
> first one could shove a "ttl" or "expiration" key into the protocol,
> which the cache could then learn to respect.

Composing helpers like this is how I encourage users to configure
git-credential-oauth [1][2]. Note that the storage helper should come
*before* the generator, so that `credential fill` finds a stored
credential before it generates a fresh credential.

> the first one could shove a "ttl" or "expiration" key into the protocol,
> which the cache could then learn to respect.
>
> But we never merged such a thing, and in practice I think people would
> just implement both parts as a single helper for simplicity.

Composing helpers has the advantage that the user can choose their
preferred storage. Generated credentials aren't necessarily short
lived. GitHub OAuth tokens, for example, are good for at least one
year [3].

[1] https://lore.kernel.org/git/CAGJzqs=+fCQzkDX53H8Mz-DjXicVVgRmmzPjkatSiOpYO7wGGA@mail.gmail.com/
[2] https://github.com/hickford/git-credential-oauth
[3] https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/token-expiration-and-revocation#token-expired-due-to-lack-of-use

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[Jeff King]

On Sat, Nov 12, 2022 at 02:21:24AM +0000, M Hickford wrote:

> On Tue, 25 Oct 2022 at 00:59, Jeff King <peff@peff.net> wrote:
> >
> > On Mon, Oct 24, 2022 at 07:57:48AM +0000, M Hickford via GitGitGadget wrote:
> >
> > > It was previously unclear how unrecognised attributes are handled.
> >
> > Yeah, this was always part of the intended behavior, but I agree we did
> > not say it very explicitly (aside from an in-code comment!). Both the
> > intent and content of your patch look good to me.
> 
> Thanks. What happens next? I should look for this change in the seen
> branch? https://git-scm.com/docs/MyFirstContribution#after-approval

Usually the maintainer would pick it up, it would end up in seen, then
eventually 'next', and then eventually 'master'. You can check the
periodic "What's Cooking" messages from the maintainer to see more
discussion of various topic branches.

In this case, though, I don't see any indication that the maintainer
picked saw it. It sometimes happens that a topic is simply overlooked,
even if it received positive reviews.

The usual thing to do is repost it, cc-ing the maintainer. I've also
cc'd the interim maintainer here, so that may get things moving. :)

> > We did discuss patches a long time ago that would let Git carry
> > arbitrary keys between helpers, even if Git itself didn't understand it.
> > One of the intended uses was to let helpers talk to each other about
> > TTLs. So if you had say:
> >
> >   [credential]
> >   helper = generate-some-token
> >   helper = cache
> >
> > where the first helper generates a token, and the second caches it, the
> > first one could shove a "ttl" or "expiration" key into the protocol,
> > which the cache could then learn to respect.
> 
> Composing helpers like this is how I encourage users to configure
> git-credential-oauth [1][2]. Note that the storage helper should come
> *before* the generator, so that `credential fill` finds a stored
> credential before it generates a fresh credential.

Right, it's been a while since I've constructed an example like this. ;)

What you're doing works fine with the code as-is; you just can't carry
extra data (like a ttl) between the two.

The thread I linked earlier also discusses (in the very top-level patch)
a change in behavior that would break the flow you're relying on here
(because it may unexpectedly propagate credentials between helpers). But
I don't think anybody is interested in pursuing that, and it has been 10
years now.

> > But we never merged such a thing, and in practice I think people would
> > just implement both parts as a single helper for simplicity.
> 
> Composing helpers has the advantage that the user can choose their
> preferred storage. Generated credentials aren't necessarily short
> lived. GitHub OAuth tokens, for example, are good for at least one
> year [3].

Yeah, the composability was one of the goals of the system. I just think
in practice that not many people use it. You can also compose outside of
Git (I think the thread I linked earlier has an example of a wrapper
that does so), but again, I don't think anybody really does so in
practice.

I agree for GitHub's tokens that the times involved make auto-expiration
not that important. The example back in that thread was something more
time-limited (like minutes or hours). I don't know how often that kind
of things is in the wild.

-Peff

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[M Hickford]

On Sat, 12 Nov 2022 at 16:47, Jeff King <peff@peff.net> wrote:
> > > We did discuss patches a long time ago that would let Git carry
> > > arbitrary keys between helpers, even if Git itself didn't understand it.
> > > One of the intended uses was to let helpers talk to each other about
> > > TTLs. So if you had say:
> > >
> > >   [credential]
> > >   helper = generate-some-token
> > >   helper = cache
> > >
> > > where the first helper generates a token, and the second caches it, the
> > > first one could shove a "ttl" or "expiration" key into the protocol,
> > > which the cache could then learn to respect.
> >
>
> What you're doing works fine with the code as-is; you just can't carry
> extra data (like a ttl) between the two.

FWIW I have a draft patch that adds password_expiry_utc and
oauth_refresh_token attributes to credential
https://github.com/gitgitgadget/git/pull/1394 introducing expiry logic
in the credential layer. I'll share a RFC sometime in future.

> I agree for GitHub's tokens that the times involved make auto-expiration
> not that important. The example back in that thread was something more
> time-limited (like minutes or hours). I don't know how often that kind
> of things is in the wild.

GitLab OAuth tokens expire after 2 hours (the refresh tokens are valid
longer). This is a security improvement over long-lived tokens.

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[Taylor Blau]

On Sat, Nov 12, 2022 at 11:47:30AM -0500, Jeff King wrote:
> On Sat, Nov 12, 2022 at 02:21:24AM +0000, M Hickford wrote:
>
> > On Tue, 25 Oct 2022 at 00:59, Jeff King <peff@peff.net> wrote:
> > >
> > > On Mon, Oct 24, 2022 at 07:57:48AM +0000, M Hickford via GitGitGadget wrote:
> > >
> > > > It was previously unclear how unrecognised attributes are handled.
> > >
> > > Yeah, this was always part of the intended behavior, but I agree we did
> > > not say it very explicitly (aside from an in-code comment!). Both the
> > > intent and content of your patch look good to me.
> >
> > Thanks. What happens next? I should look for this change in the seen
> > branch? https://git-scm.com/docs/MyFirstContribution#after-approval
>
> Usually the maintainer would pick it up, it would end up in seen, then
> eventually 'next', and then eventually 'master'. You can check the
> periodic "What's Cooking" messages from the maintainer to see more
> discussion of various topic branches.
>
> In this case, though, I don't see any indication that the maintainer
> picked saw it. It sometimes happens that a topic is simply overlooked,
> even if it received positive reviews.
>
> The usual thing to do is repost it, cc-ing the maintainer. I've also
> cc'd the interim maintainer here, so that may get things moving. :)

I think that Junio may have missed it when picking up topics. But it
happened before I start combing the list more finely, so I had already
purged it from my inbox without picking it up, either.

I'll take a look now and queue it to start merging down. Thanks for the
nudge.

Thanks,
Taylor

Re: [PATCH] docs: clarify that credential discards unrecognised attributes

[Jeff King]

On Sat, Nov 12, 2022 at 07:08:42PM +0000, M Hickford wrote:

> On Sat, 12 Nov 2022 at 16:47, Jeff King <peff@peff.net> wrote:
> > > > We did discuss patches a long time ago that would let Git carry
> > > > arbitrary keys between helpers, even if Git itself didn't understand it.
> > > > One of the intended uses was to let helpers talk to each other about
> > > > TTLs. So if you had say:
> > > >
> > > >   [credential]
> > > >   helper = generate-some-token
> > > >   helper = cache
> > > >
> > > > where the first helper generates a token, and the second caches it, the
> > > > first one could shove a "ttl" or "expiration" key into the protocol,
> > > > which the cache could then learn to respect.
> > >
> >
> > What you're doing works fine with the code as-is; you just can't carry
> > extra data (like a ttl) between the two.
> 
> FWIW I have a draft patch that adds password_expiry_utc and
> oauth_refresh_token attributes to credential
> https://github.com/gitgitgadget/git/pull/1394 introducing expiry logic
> in the credential layer. I'll share a RFC sometime in future.

Neat.

I'm not _totally_ opposed to introducing these as something Git
understands, but I think it makes more sense to just teach Git to relay
unknown entries between helpers.

The oauth thing is going to be very helper specific, and not something I
think Git would ever do anything with itself.

In theory Git might care about expiration, but in practice I think it
doesn't. It's very unlikely for a token to expire in the course of Git
using it. It's only much later, when we ask for it back, that a helper
will notice it's expired. Git could save the helper some work by
noticing this on read, but since the helper has to learn to store and
report the expiration in the first place, not much is gained.

And in the case of something like credential-cache, we want to do more
than just store; we'd actually drop the credential entirely (and maybe
even cause the daemon to exit) if it expires before the usual timeout.

-Peff

Thanks

[Tatyana Polunin]

Thanks!