Temporary credentials timeout during long operations

Temporary credentials timeout during long operations

[Peterson, Alex]

Greetings,  recently I've run into an issue where long push operations fail when using a credential helper like the AWS CodeCommit helper because it provides time-limited credentials which expire during the operation.  Is there an existing method to enforce refreshing credentials from a credential helper on the client or should I go down the path of creating a credential.timeout feature that automatically refills credentials from helpers during long operations to avoid this issue?

Steps to reproduce:
1. Create a repository with LFS
2. Add a large file that will take a long time to push (long enough for the credentials to expire)
3. Enable a helper that provides temporary credentials 
4. Set a remote that uses HTTPS and git push

All initial HTTPS commands authenticate OK and the LFS file uploads fine but by the time it is done, the credentials have expired so the final git-receive-pack fails and the entire push operation fails.

> POST /v1/repos/example/git-receive-pack HTTP/1.1
< HTTP/1.1 403 
error: RPC failed; HTTP 403 curl 22 The requested URL returned error: 403 
fatal: the remote end hung up unexpectedly 

For testing purposes I modified post_rpc() so that it always requests new credentials before every HTTPS request, and was able to push any number of large files.

SSH works fine (with keepalives enabled) and so does HTTPS with a permanent username/password, it's just the credential helpers that provide time-limited credentials.

Thoughts?

Re: Temporary credentials timeout during long operations

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

[Please keep the list in CC.  Other people may have important
contributions to the discussion, and due to weather, I may be offline at
some point in the future and be unable to respond.]

On 2020-08-26 at 00:18:05, Peterson, Alex wrote:
> Hi Brian,
> 
> Unfortunately, even if the server returns a 401, git will retry but with the old expired credentials which will fail.  I believe it is because of this line that checks if a username/password already exists (which it does)
> https://github.com/git/git/blob/07d8ea56f2ecb64b75b92264770c0a664231ce17/credential.c#L338
> 
> In my test I cleared the username and password to force it to re-request credentials and that worked OK.

Ah, yes.  In that case, it looks like we call credential_reject and then
return HTTP_NOAUTH.  I think the assumption is that the credential
helper returns a consistent set of credentials and once we've told the
credential helper to reject them, then the user can push again and be
prompted for new credentials.

I would be open to seeing a patch which, the first time through,
returned HTTP_REAUTH.  We wouldn't want to do that indefinitely, since
that would mean that the user would get stuck in a loop if the
credentials were wrong.

I will say that my gut tells me that it's generally a reasonable
assumption that credentials are valid for the life of a push, whatever
that is, so while I'm not opposed to seeing a patch to improve this, I'm
not especially sympathetic to using credentials that have such a short
lifetime that this occurs, even if I am in general in support of
short-lived credentials.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]