[PATCH 0/4] osxkeychain: bring in line with other credential helpers

[PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Bo Anderson via GitGitGadget]

git-credential-osxkeychain has largely fallen behind other external
credential helpers in the features it supports, and hasn't received any
functional changes since 2013. As it stood, osxkeychain failed seven tests
in the external credential helper test suite:

not ok 8 - helper (osxkeychain) overwrites on store
not ok 9 - helper (osxkeychain) can forget host
not ok 11 - helper (osxkeychain) does not erase a password distinct from input
not ok 15 - helper (osxkeychain) erases all matching credentials
not ok 18 - helper (osxkeychain) gets password_expiry_utc
not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
not ok 21 - helper (osxkeychain) gets oauth_refresh_token


osxkeychain also made use of macOS APIs that had been deprecated since 2014.
Replacement API was able to be used without regressing the minimum supported
macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer
in osxkeychain, 2023-05-01).

After this set of patches, osxkeychain passes all tests in the external
credential helper test suite.

Bo Anderson (4):
  osxkeychain: replace deprecated SecKeychain API
  osxkeychain: erase all matching credentials
  osxkeychain: erase matching passwords only
  osxkeychain: store new attributes

 contrib/credential/osxkeychain/Makefile       |   3 +-
 .../osxkeychain/git-credential-osxkeychain.c  | 376 ++++++++++++++----
 2 files changed, 310 insertions(+), 69 deletions(-)


base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1667%2FBo98%2Fosxkeychain-update-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1667/Bo98/osxkeychain-update-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1667
-- 
gitgitgadget

[PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

[Bo Anderson via GitGitGadget]

From: Bo Anderson <mail@boanderson.me>

The SecKeychain API was deprecated in macOS 10.10, nearly 10 years ago.
The replacement SecItem API however is available as far back as macOS
10.6.

While supporting older macOS was perhaps prevously a concern,
git-credential-osxkeychain already requires a minimum of macOS 10.7
since 5747c8072b (contrib/credential: avoid fixed-size buffer in
osxkeychain, 2023-05-01) so using the newer API should not regress the
range of macOS versions supported.

Adapting to use the newer SecItem API also happens to fix two test
failures in osxkeychain:

    8 - helper (osxkeychain) overwrites on store
    9 - helper (osxkeychain) can forget host

The new API is compatible with credentials saved with the older API.

Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 contrib/credential/osxkeychain/Makefile       |   3 +-
 .../osxkeychain/git-credential-osxkeychain.c  | 265 +++++++++++++-----
 2 files changed, 199 insertions(+), 69 deletions(-)

diff --git a/contrib/credential/osxkeychain/Makefile b/contrib/credential/osxkeychain/Makefile
index 4b3a08a2bac..238f5f8c36f 100644
--- a/contrib/credential/osxkeychain/Makefile
+++ b/contrib/credential/osxkeychain/Makefile
@@ -8,7 +8,8 @@ CFLAGS = -g -O2 -Wall
 -include ../../../config.mak
 
 git-credential-osxkeychain: git-credential-osxkeychain.o
-	$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) -Wl,-framework -Wl,Security
+	$(CC) $(CFLAGS) -o $@ $< $(LDFLAGS) \
+		-framework Security -framework CoreFoundation
 
 git-credential-osxkeychain.o: git-credential-osxkeychain.c
 	$(CC) -c $(CFLAGS) $<
diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index 5f2e5f16c88..dc294ae944a 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -3,14 +3,39 @@
 #include <stdlib.h>
 #include <Security/Security.h>
 
-static SecProtocolType protocol;
-static char *host;
-static char *path;
-static char *username;
-static char *password;
-static UInt16 port;
-
-__attribute__((format (printf, 1, 2)))
+#define ENCODING kCFStringEncodingUTF8
+static CFStringRef protocol; /* Stores constant strings - not memory managed */
+static CFStringRef host;
+static CFStringRef path;
+static CFStringRef username;
+static CFDataRef password;
+static CFNumberRef port;
+
+static void clear_credential(void)
+{
+	if (host) {
+		CFRelease(host);
+		host = NULL;
+	}
+	if (path) {
+		CFRelease(path);
+		path = NULL;
+	}
+	if (username) {
+		CFRelease(username);
+		username = NULL;
+	}
+	if (password) {
+		CFRelease(password);
+		password = NULL;
+	}
+	if (port) {
+		CFRelease(port);
+		port = NULL;
+	}
+}
+
+__attribute__((format (printf, 1, 2), __noreturn__))
 static void die(const char *err, ...)
 {
 	char msg[4096];
@@ -19,70 +44,135 @@ static void die(const char *err, ...)
 	vsnprintf(msg, sizeof(msg), err, params);
 	fprintf(stderr, "%s\n", msg);
 	va_end(params);
+	clear_credential();
 	exit(1);
 }
 
-static void *xstrdup(const char *s1)
+static void *xmalloc(size_t len)
 {
-	void *ret = strdup(s1);
+	void *ret = malloc(len);
 	if (!ret)
 		die("Out of memory");
 	return ret;
 }
 
-#define KEYCHAIN_ITEM(x) (x ? strlen(x) : 0), x
-#define KEYCHAIN_ARGS \
-	NULL, /* default keychain */ \
-	KEYCHAIN_ITEM(host), \
-	0, NULL, /* account domain */ \
-	KEYCHAIN_ITEM(username), \
-	KEYCHAIN_ITEM(path), \
-	port, \
-	protocol, \
-	kSecAuthenticationTypeDefault
-
-static void write_item(const char *what, const char *buf, int len)
+static CFDictionaryRef create_dictionary(CFAllocatorRef allocator, ...)
+{
+	va_list args;
+	const void *key;
+	CFMutableDictionaryRef result;
+
+	result = CFDictionaryCreateMutable(allocator,
+					   0,
+					   &kCFTypeDictionaryKeyCallBacks,
+					   &kCFTypeDictionaryValueCallBacks);
+
+
+	va_start(args, allocator);
+	while ((key = va_arg(args, const void *)) != NULL) {
+		const void *value;
+		value = va_arg(args, const void *);
+		if (value)
+			CFDictionarySetValue(result, key, value);
+	}
+	va_end(args);
+
+	return result;
+}
+
+#define CREATE_SEC_ATTRIBUTES(...) \
+	create_dictionary(kCFAllocatorDefault, \
+			  kSecClass, kSecClassInternetPassword, \
+			  kSecAttrServer, host, \
+			  kSecAttrAccount, username, \
+			  kSecAttrPath, path, \
+			  kSecAttrPort, port, \
+			  kSecAttrProtocol, protocol, \
+			  kSecAttrAuthenticationType, \
+			  kSecAttrAuthenticationTypeDefault, \
+			  __VA_ARGS__);
+
+static void write_item(const char *what, const char *buf, size_t len)
 {
 	printf("%s=", what);
 	fwrite(buf, 1, len, stdout);
 	putchar('\n');
 }
 
-static void find_username_in_item(SecKeychainItemRef item)
+static void find_username_in_item(CFDictionaryRef item)
 {
-	SecKeychainAttributeList list;
-	SecKeychainAttribute attr;
+	CFStringRef account_ref;
+	char *username_buf;
+	CFIndex buffer_len;
 
-	list.count = 1;
-	list.attr = &attr;
-	attr.tag = kSecAccountItemAttr;
+	account_ref = CFDictionaryGetValue(item, kSecAttrAccount);
+	if (!account_ref)
+	{
+		write_item("username", "", 0);
+		return;
+	}
 
-	if (SecKeychainItemCopyContent(item, NULL, &list, NULL, NULL))
+	username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
+	if (username_buf)
+	{
+		write_item("username", username_buf, strlen(username_buf));
 		return;
+	}
 
-	write_item("username", attr.data, attr.length);
-	SecKeychainItemFreeContent(&list, NULL);
+	/* If we can't get a CString pointer then
+	 * we need to allocate our own buffer */
+	buffer_len = CFStringGetMaximumSizeForEncoding(
+			CFStringGetLength(account_ref), ENCODING) + 1;
+	username_buf = xmalloc(buffer_len);
+	if (CFStringGetCString(account_ref,
+				username_buf,
+				buffer_len,
+				ENCODING)) {
+		write_item("username", username_buf, buffer_len - 1);
+	}
+	free(username_buf);
 }
 
-static void find_internet_password(void)
+static OSStatus find_internet_password(void)
 {
-	void *buf;
-	UInt32 len;
-	SecKeychainItemRef item;
+	CFDictionaryRef attrs;
+	CFDictionaryRef item;
+	CFDataRef data;
+	OSStatus result;
 
-	if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, &len, &buf, &item))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitOne,
+				      kSecReturnAttributes, kCFBooleanTrue,
+				      kSecReturnData, kCFBooleanTrue,
+				      NULL);
+	result = SecItemCopyMatching(attrs, (CFTypeRef *)&item);
+	if (result) {
+		goto out;
+	}
 
-	write_item("password", buf, len);
+	data = CFDictionaryGetValue(item, kSecValueData);
+
+	write_item("password",
+		   (const char *)CFDataGetBytePtr(data),
+		   CFDataGetLength(data));
 	if (!username)
 		find_username_in_item(item);
 
-	SecKeychainItemFreeContent(NULL, buf);
+	CFRelease(item);
+
+out:
+	CFRelease(attrs);
+
+	/* We consider not found to not be an error */
+	if (result == errSecItemNotFound)
+		result = errSecSuccess;
+
+	return result;
 }
 
-static void delete_internet_password(void)
+static OSStatus delete_internet_password(void)
 {
-	SecKeychainItemRef item;
+	CFDictionaryRef attrs;
+	OSStatus result;
 
 	/*
 	 * Require at least a protocol and host for removal, which is what git
@@ -90,25 +180,42 @@ static void delete_internet_password(void)
 	 * Keychain manager.
 	 */
 	if (!protocol || !host)
-		return;
+		return -1;
 
-	if (SecKeychainFindInternetPassword(KEYCHAIN_ARGS, 0, NULL, &item))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(NULL);
+	result = SecItemDelete(attrs);
+	CFRelease(attrs);
+
+	/* We consider not found to not be an error */
+	if (result == errSecItemNotFound)
+		result = errSecSuccess;
 
-	SecKeychainItemDelete(item);
+	return result;
 }
 
-static void add_internet_password(void)
+static OSStatus add_internet_password(void)
 {
+	CFDictionaryRef attrs;
+	OSStatus result;
+
 	/* Only store complete credentials */
 	if (!protocol || !host || !username || !password)
-		return;
+		return -1;
 
-	if (SecKeychainAddInternetPassword(
-	      KEYCHAIN_ARGS,
-	      KEYCHAIN_ITEM(password),
-	      NULL))
-		return;
+	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
+				      NULL);
+
+	result = SecItemAdd(attrs, NULL);
+	if (result == errSecDuplicateItem) {
+		CFDictionaryRef query;
+		query = CREATE_SEC_ATTRIBUTES(NULL);
+		result = SecItemUpdate(query, attrs);
+		CFRelease(query);
+	}
+
+	CFRelease(attrs);
+
+	return result;
 }
 
 static void read_credential(void)
@@ -131,36 +238,52 @@ static void read_credential(void)
 
 		if (!strcmp(buf, "protocol")) {
 			if (!strcmp(v, "imap"))
-				protocol = kSecProtocolTypeIMAP;
+				protocol = kSecAttrProtocolIMAP;
 			else if (!strcmp(v, "imaps"))
-				protocol = kSecProtocolTypeIMAPS;
+				protocol = kSecAttrProtocolIMAPS;
 			else if (!strcmp(v, "ftp"))
-				protocol = kSecProtocolTypeFTP;
+				protocol = kSecAttrProtocolFTP;
 			else if (!strcmp(v, "ftps"))
-				protocol = kSecProtocolTypeFTPS;
+				protocol = kSecAttrProtocolFTPS;
 			else if (!strcmp(v, "https"))
-				protocol = kSecProtocolTypeHTTPS;
+				protocol = kSecAttrProtocolHTTPS;
 			else if (!strcmp(v, "http"))
-				protocol = kSecProtocolTypeHTTP;
+				protocol = kSecAttrProtocolHTTP;
 			else if (!strcmp(v, "smtp"))
-				protocol = kSecProtocolTypeSMTP;
-			else /* we don't yet handle other protocols */
+				protocol = kSecAttrProtocolSMTP;
+			else {
+				/* we don't yet handle other protocols */
+				clear_credential();
 				exit(0);
+			}
 		}
 		else if (!strcmp(buf, "host")) {
 			char *colon = strchr(v, ':');
 			if (colon) {
+				UInt16 port_i;
 				*colon++ = '\0';
-				port = atoi(colon);
+				port_i = atoi(colon);
+				port = CFNumberCreate(kCFAllocatorDefault,
+						      kCFNumberShortType,
+						      &port_i);
 			}
-			host = xstrdup(v);
+			host = CFStringCreateWithCString(kCFAllocatorDefault,
+							 v,
+							 ENCODING);
 		}
 		else if (!strcmp(buf, "path"))
-			path = xstrdup(v);
+			path = CFStringCreateWithCString(kCFAllocatorDefault,
+							 v,
+							 ENCODING);
 		else if (!strcmp(buf, "username"))
-			username = xstrdup(v);
+			username = CFStringCreateWithCString(
+					kCFAllocatorDefault,
+					v,
+					ENCODING);
 		else if (!strcmp(buf, "password"))
-			password = xstrdup(v);
+			password = CFDataCreate(kCFAllocatorDefault,
+						(UInt8 *)v,
+						strlen(v));
 		/*
 		 * Ignore other lines; we don't know what they mean, but
 		 * this future-proofs us when later versions of git do
@@ -173,6 +296,7 @@ static void read_credential(void)
 
 int main(int argc, const char **argv)
 {
+	OSStatus result = 0;
 	const char *usage =
 		"usage: git credential-osxkeychain <get|store|erase>";
 
@@ -182,12 +306,17 @@ int main(int argc, const char **argv)
 	read_credential();
 
 	if (!strcmp(argv[1], "get"))
-		find_internet_password();
+		result = find_internet_password();
 	else if (!strcmp(argv[1], "store"))
-		add_internet_password();
+		result = add_internet_password();
 	else if (!strcmp(argv[1], "erase"))
-		delete_internet_password();
+		result = delete_internet_password();
 	/* otherwise, ignore unknown action */
 
+	if (result)
+		die("failed to %s: %d", argv[1], (int)result);
+
+	clear_credential();
+
 	return 0;
 }
-- 
gitgitgadget

[PATCH 2/4] osxkeychain: erase all matching credentials

[Bo Anderson via GitGitGadget]

From: Bo Anderson <mail@boanderson.me>

Other credential managers erased all matching credentials, as indicated
by a test case that osxkeychain failed:

    15 - helper (osxkeychain) erases all matching credentials

Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 contrib/credential/osxkeychain/git-credential-osxkeychain.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index dc294ae944a..e9cee3aed45 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -182,7 +182,8 @@ static OSStatus delete_internet_password(void)
 	if (!protocol || !host)
 		return -1;
 
-	attrs = CREATE_SEC_ATTRIBUTES(NULL);
+	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitAll,
+				      NULL);
 	result = SecItemDelete(attrs);
 	CFRelease(attrs);
 
-- 
gitgitgadget

[PATCH 3/4] osxkeychain: erase matching passwords only

[Bo Anderson via GitGitGadget]

From: Bo Anderson <mail@boanderson.me>

Other credential helpers support deleting credentials that match a
specified password. See 7144dee3ec (credential/libsecret: erase matching
creds only, 2023-07-26) and cb626f8e5c (credential/wincred: erase
matching creds only, 2023-07-26).

Support this in osxkeychain too by extracting, decrypting and comparing
the stored password before deleting.

Fixes the following test failure with osxkeychain:

    11 - helper (osxkeychain) does not erase a password distinct from
    input

Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 .../osxkeychain/git-credential-osxkeychain.c  | 56 ++++++++++++++++++-
 1 file changed, 55 insertions(+), 1 deletion(-)

diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index e9cee3aed45..9e742796336 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -169,9 +169,55 @@ static OSStatus find_internet_password(void)
 	return result;
 }
 
+static OSStatus delete_ref(const void *itemRef)
+{
+	CFArrayRef item_ref_list;
+	CFDictionaryRef delete_query;
+	OSStatus result;
+
+	item_ref_list = CFArrayCreate(kCFAllocatorDefault,
+				      &itemRef,
+				      1,
+				      &kCFTypeArrayCallBacks);
+	delete_query = create_dictionary(kCFAllocatorDefault,
+					 kSecClass, kSecClassInternetPassword,
+					 kSecMatchItemList, item_ref_list,
+					 NULL);
+
+	if (password) {
+		/* We only want to delete items with a matching password */
+		CFIndex capacity;
+		CFMutableDictionaryRef query;
+		CFDataRef data;
+
+		capacity = CFDictionaryGetCount(delete_query) + 1;
+		query = CFDictionaryCreateMutableCopy(kCFAllocatorDefault,
+						      capacity,
+						      delete_query);
+		CFDictionarySetValue(query, kSecReturnData, kCFBooleanTrue);
+		result = SecItemCopyMatching(query, (CFTypeRef *)&data);
+		if (!result) {
+			if (CFEqual(data, password))
+				result = SecItemDelete(delete_query);
+
+			CFRelease(data);
+		}
+
+		CFRelease(query);
+	} else {
+		result = SecItemDelete(delete_query);
+	}
+
+	CFRelease(delete_query);
+	CFRelease(item_ref_list);
+
+	return result;
+}
+
 static OSStatus delete_internet_password(void)
 {
 	CFDictionaryRef attrs;
+	CFArrayRef refs;
 	OSStatus result;
 
 	/*
@@ -183,10 +229,18 @@ static OSStatus delete_internet_password(void)
 		return -1;
 
 	attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitAll,
+				      kSecReturnRef, kCFBooleanTrue,
 				      NULL);
-	result = SecItemDelete(attrs);
+	result = SecItemCopyMatching(attrs, (CFTypeRef *)&refs);
 	CFRelease(attrs);
 
+	if (!result) {
+		for (CFIndex i = 0; !result && i < CFArrayGetCount(refs); i++)
+			result = delete_ref(CFArrayGetValueAtIndex(refs, i));
+
+		CFRelease(refs);
+	}
+
 	/* We consider not found to not be an error */
 	if (result == errSecItemNotFound)
 		result = errSecSuccess;
-- 
gitgitgadget

[PATCH 4/4] osxkeychain: store new attributes

[Bo Anderson via GitGitGadget]

From: Bo Anderson <mail@boanderson.me>

d208bfdfef (credential: new attribute password_expiry_utc, 2023-02-18)
and a5c76569e7 (credential: new attribute oauth_refresh_token,
2023-04-21) introduced new credential attributes but support was missing
from git-credential-osxkeychain.

Support these attributes by appending the data to the password in the
keychain, separated by line breaks. Line breaks cannot appear in a git
credential password so it is an appropriate separator.

Fixes the remaining test failures with osxkeychain:

    18 - helper (osxkeychain) gets password_expiry_utc
    19 - helper (osxkeychain) overwrites when password_expiry_utc
    changes
    21 - helper (osxkeychain) gets oauth_refresh_token

Signed-off-by: Bo Anderson <mail@boanderson.me>
---
 .../osxkeychain/git-credential-osxkeychain.c  | 68 +++++++++++++++++--
 1 file changed, 62 insertions(+), 6 deletions(-)

diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
index 9e742796336..6a40917b1ef 100644
--- a/contrib/credential/osxkeychain/git-credential-osxkeychain.c
+++ b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
@@ -6,10 +6,12 @@
 #define ENCODING kCFStringEncodingUTF8
 static CFStringRef protocol; /* Stores constant strings - not memory managed */
 static CFStringRef host;
+static CFNumberRef port;
 static CFStringRef path;
 static CFStringRef username;
 static CFDataRef password;
-static CFNumberRef port;
+static CFDataRef password_expiry_utc;
+static CFDataRef oauth_refresh_token;
 
 static void clear_credential(void)
 {
@@ -17,6 +19,10 @@ static void clear_credential(void)
 		CFRelease(host);
 		host = NULL;
 	}
+	if (port) {
+		CFRelease(port);
+		port = NULL;
+	}
 	if (path) {
 		CFRelease(path);
 		path = NULL;
@@ -29,12 +35,18 @@ static void clear_credential(void)
 		CFRelease(password);
 		password = NULL;
 	}
-	if (port) {
-		CFRelease(port);
-		port = NULL;
+	if (password_expiry_utc) {
+		CFRelease(password_expiry_utc);
+		password_expiry_utc = NULL;
+	}
+	if (oauth_refresh_token) {
+		CFRelease(oauth_refresh_token);
+		oauth_refresh_token = NULL;
 	}
 }
 
+#define STRING_WITH_LENGTH(s) s, sizeof(s) - 1
+
 __attribute__((format (printf, 1, 2), __noreturn__))
 static void die(const char *err, ...)
 {
@@ -197,9 +209,27 @@ static OSStatus delete_ref(const void *itemRef)
 		CFDictionarySetValue(query, kSecReturnData, kCFBooleanTrue);
 		result = SecItemCopyMatching(query, (CFTypeRef *)&data);
 		if (!result) {
-			if (CFEqual(data, password))
+			CFDataRef kc_password;
+			const UInt8 *raw_data;
+			const UInt8 *line;
+
+			/* Don't match appended metadata */
+			raw_data = CFDataGetBytePtr(data);
+			line = memchr(raw_data, '\n', CFDataGetLength(data));
+			if (line)
+				kc_password = CFDataCreateWithBytesNoCopy(
+						kCFAllocatorDefault,
+						raw_data,
+						line - raw_data,
+						kCFAllocatorNull);
+			else
+				kc_password = data;
+
+			if (CFEqual(kc_password, password))
 				result = SecItemDelete(delete_query);
 
+			if (line)
+				CFRelease(kc_password);
 			CFRelease(data);
 		}
 
@@ -250,6 +280,7 @@ static OSStatus delete_internet_password(void)
 
 static OSStatus add_internet_password(void)
 {
+	CFMutableDataRef data;
 	CFDictionaryRef attrs;
 	OSStatus result;
 
@@ -257,7 +288,23 @@ static OSStatus add_internet_password(void)
 	if (!protocol || !host || !username || !password)
 		return -1;
 
-	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
+	data = CFDataCreateMutableCopy(kCFAllocatorDefault, 0, password);
+	if (password_expiry_utc) {
+		CFDataAppendBytes(data,
+		    (const UInt8 *)STRING_WITH_LENGTH("\npassword_expiry_utc="));
+		CFDataAppendBytes(data,
+				  CFDataGetBytePtr(password_expiry_utc),
+				  CFDataGetLength(password_expiry_utc));
+	}
+	if (oauth_refresh_token) {
+		CFDataAppendBytes(data,
+		    (const UInt8 *)STRING_WITH_LENGTH("\noauth_refresh_token="));
+		CFDataAppendBytes(data,
+				  CFDataGetBytePtr(oauth_refresh_token),
+				  CFDataGetLength(oauth_refresh_token));
+	}
+
+	attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, data,
 				      NULL);
 
 	result = SecItemAdd(attrs, NULL);
@@ -268,6 +315,7 @@ static OSStatus add_internet_password(void)
 		CFRelease(query);
 	}
 
+	CFRelease(data);
 	CFRelease(attrs);
 
 	return result;
@@ -339,6 +387,14 @@ static void read_credential(void)
 			password = CFDataCreate(kCFAllocatorDefault,
 						(UInt8 *)v,
 						strlen(v));
+		else if (!strcmp(buf, "password_expiry_utc"))
+			password_expiry_utc = CFDataCreate(kCFAllocatorDefault,
+							   (UInt8 *)v,
+							   strlen(v));
+		else if (!strcmp(buf, "oauth_refresh_token"))
+			oauth_refresh_token = CFDataCreate(kCFAllocatorDefault,
+							   (UInt8 *)v,
+							   strlen(v));
 		/*
 		 * Ignore other lines; we don't know what they mean, but
 		 * this future-proofs us when later versions of git do
-- 
gitgitgadget

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

[Eric Sunshine]

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> The SecKeychain API was deprecated in macOS 10.10, nearly 10 years ago.
> The replacement SecItem API however is available as far back as macOS
> 10.6.
>
> While supporting older macOS was perhaps prevously a concern,
> git-credential-osxkeychain already requires a minimum of macOS 10.7
> since 5747c8072b (contrib/credential: avoid fixed-size buffer in
> osxkeychain, 2023-05-01) so using the newer API should not regress the
> range of macOS versions supported.
>
> Adapting to use the newer SecItem API also happens to fix two test
> failures in osxkeychain:
>
>     8 - helper (osxkeychain) overwrites on store
>     9 - helper (osxkeychain) can forget host
>
> The new API is compatible with credentials saved with the older API.
>
> Signed-off-by: Bo Anderson <mail@boanderson.me>

I haven't studied the SecItem API, so I can't comment on the meat of
the patch, but I can make a few generic observations...

> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
> @@ -3,14 +3,39 @@
> -__attribute__((format (printf, 1, 2)))
> +#define ENCODING kCFStringEncodingUTF8
> +static CFStringRef protocol; /* Stores constant strings - not memory managed */
> +static CFStringRef host;
> [...]
> +
> +static void clear_credential(void)
> +{
> +       if (host) {
> +               CFRelease(host);
> +               host = NULL;
> +       }
> +       [...]
> +}
> +
> +__attribute__((format (printf, 1, 2), __noreturn__))

The addition of `__noreturn__` to the `__attribute__` seems unrelated
to the stated purpose of this patch. As such, it typically would be
placed in its own patch. If it really is too minor for a separate
patch, mentioning it in the commit message as a "While at it..." would
be helpful.

> @@ -19,70 +44,135 @@ static void die(const char *err, ...)
> +static CFDictionaryRef create_dictionary(CFAllocatorRef allocator, ...)
> +{
> +       va_list args;
> +       const void *key;
> +       CFMutableDictionaryRef result;
> +
> +       result = CFDictionaryCreateMutable(allocator,
> +                                          0,
> +                                          &kCFTypeDictionaryKeyCallBacks,
> +                                          &kCFTypeDictionaryValueCallBacks);
> +
> +

Style: one blank line is preferred over two

> +       va_start(args, allocator);
> +       while ((key = va_arg(args, const void *)) != NULL) {
> +               const void *value;
> +               value = va_arg(args, const void *);
> +               if (value)
> +                       CFDictionarySetValue(result, key, value);
> +       }
> +       va_end(args);

A couple related comments...

If va_arg() ever returns NULL for `value`, the next iteration of the
loop will call va_arg() again, but calling va_arg() again after it has
already returned NULL is likely undefined behavior. At minimum, I
would have expected this to be written as:

    while (...) {
        ...
        if (!value)
            break;
        CFDictionarySetValue(...);
    }

However, isn't it a programmer error if va_arg() returns NULL for
`value`? If so, I'd think we'd want to scream loudly about that rather
than silently ignoring it. So, perhaps something like this:

    while (...) {
        ...
        if (!value) {
            fprintf(stderr, "BUG: ...");
            abort();
        }
        CFDictionarySetValue(...);
   }

Or, perhaps just call the existing die() function in this file with a
suitable "BUG ..." message.

> +static void find_username_in_item(CFDictionaryRef item)
>  {
> +       CFStringRef account_ref;
> +       char *username_buf;
> +       CFIndex buffer_len;
>
> +       account_ref = CFDictionaryGetValue(item, kSecAttrAccount);
> +       if (!account_ref)
> +       {
> +               write_item("username", "", 0);
> +               return;
> +       }

Style: opening brace sticks to the `if` line:

    if !(account_ref) {
        ...
    }

Same comment applies to the `if` below.

> +       username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
> +       if (username_buf)
> +       {
> +               write_item("username", username_buf, strlen(username_buf));
>                 return;
> +       }

According to the documentation for CFStringGetCStringPtr(), the
returned C-string is not newly-allocated, so the caller does not have
to free it. Therefore, can `username_buf` be declared `const char *`
rather than `char *` to make it clear to readers that nothing is being
leaked here? Same comment about the `(char *)` cast.

> +       /* If we can't get a CString pointer then
> +        * we need to allocate our own buffer */

Style:

    /*
     * Multi-line comments
     * are formatted like this.
     */

> +       buffer_len = CFStringGetMaximumSizeForEncoding(
> +                       CFStringGetLength(account_ref), ENCODING) + 1;
> +       username_buf = xmalloc(buffer_len);
> +       if (CFStringGetCString(account_ref,
> +                               username_buf,
> +                               buffer_len,
> +                               ENCODING)) {
> +               write_item("username", username_buf, buffer_len - 1);
> +       }
> +       free(username_buf);

Okay, this explains why `username_buf` is declared `char *` rather
than `const char *`. Typically, when we have a situation in which a
value may or may not need freeing, we use a `to_free` variable like
this:

    const char *username_buf;
    char *to_free = NULL;
    ...
    username_buf = (const char *)CFStringGetCStringPtr(...);
    if (username_buf) {
        ...
        return;
    }
    ...
    username_buf = to_free = xmalloc(buffer_len);
    if (CFStringGetCString(...))
        ...
    free(to_free);

But that may be overkill for this simple case, and what you have here
may be "good enough" for anyone already familiar with the API and who
knows that the `return` after CFStringGetCStringPtr() isn't leaking.

> +static OSStatus find_internet_password(void)
>  {
> +       CFDictionaryRef attrs;
> +       [...]
>
> +       attrs = CREATE_SEC_ATTRIBUTES(kSecMatchLimit, kSecMatchLimitOne,
> +                                     kSecReturnAttributes, kCFBooleanTrue,
> +                                     kSecReturnData, kCFBooleanTrue,
> +                                     NULL);
> +       result = SecItemCopyMatching(attrs, (CFTypeRef *)&item);
> +       if (result) {
> +               goto out;
> +       }

We omit braces when the body is a single statement:

    if (result)
        goto out;

(Same comment applies to other code in this patch.)

> +       data = CFDictionaryGetValue(item, kSecValueData);
> +       [...]
> +
> +out:
> +       CFRelease(attrs);

Good, `attrs` is released in all cases.

> +static OSStatus add_internet_password(void)
>  {
> +       [...]
> +       attrs = CREATE_SEC_ATTRIBUTES(kSecValueData, password,
> +                                     NULL);
> +       result = SecItemAdd(attrs, NULL);
> +       if (result == errSecDuplicateItem) {
> +               CFDictionaryRef query;
> +               query = CREATE_SEC_ATTRIBUTES(NULL);
> +               result = SecItemUpdate(query, attrs);
> +               CFRelease(query);
> +       }
> +       CFRelease(attrs);
> +       return result;
>  }

Good, `attrs` and `query` are released in all cases.

Re: [PATCH 4/4] osxkeychain: store new attributes

[Eric Sunshine]

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> d208bfdfef (credential: new attribute password_expiry_utc, 2023-02-18)
> and a5c76569e7 (credential: new attribute oauth_refresh_token,
> 2023-04-21) introduced new credential attributes but support was missing
> from git-credential-osxkeychain.
> [...]
> Signed-off-by: Bo Anderson <mail@boanderson.me>
> ---
> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
> @@ -6,10 +6,12 @@
>  static CFStringRef host;
> +static CFNumberRef port;
>  static CFStringRef path;
> -static CFNumberRef port;
> @@ -17,6 +19,10 @@ static void clear_credential(void)
> +       if (port) {
> +               CFRelease(port);
> +               port = NULL;
> +       }
> @@ -29,12 +35,18 @@ static void clear_credential(void)
> -       if (port) {
> -               CFRelease(port);
> -               port = NULL;
> +       if (password_expiry_utc) {
> +               CFRelease(password_expiry_utc);
> +               password_expiry_utc = NULL;
> +       }

The relocation of `port` is unrelated to the stated purpose of this
patch. We would normally avoid this sort of "noise" change since it
obscures the "real" changes made by the patch, and would instead place
it in its own patch. That said, it's such a minor issue, I doubt that
it's worth a reroll.

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Eric Sunshine]

On Sat, Feb 17, 2024 at 6:35 PM Bo Anderson via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. [...]
>
> osxkeychain also made use of macOS APIs that had been deprecated since 2014.
> Replacement API was able to be used without regressing the minimum supported
> macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer
> in osxkeychain, 2023-05-01).

Although I'm not familiar with the SecItem API nor with the Git
keychain API, I gave this series a readthrough and left a few minor
comments. Aside from a few very minor style nits, perhaps the only
substantive comment was that patch [1/4] could do a slightly better
job of protecting against future programmer error, but even that is
minor in that it doesn't impact the functionality actually implemented
by the patch, thus may not be worth a reroll.

Overall, despite not being familiar with the APIs in question,
everything I read in the patches made sense and was cleanly
implemented. Nicely done.

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

[Bo Anderson]

> On 18 Feb 2024, at 06:08, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> I haven't studied the SecItem API, so I can't comment on the meat of
> the patch, but I can make a few generic observations...

Thanks for taking a look!

>> diff --git a/contrib/credential/osxkeychain/git-credential-osxkeychain.c b/contrib/credential/osxkeychain/git-credential-osxkeychain.c
>> @@ -3,14 +3,39 @@
>> -__attribute__((format (printf, 1, 2)))
>> +#define ENCODING kCFStringEncodingUTF8
>> +static CFStringRef protocol; /* Stores constant strings - not memory managed */
>> +static CFStringRef host;
>> [...]
>> +
>> +static void clear_credential(void)
>> +{
>> +       if (host) {
>> +               CFRelease(host);
>> +               host = NULL;
>> +       }
>> +       [...]
>> +}
>> +
>> +__attribute__((format (printf, 1, 2), __noreturn__))
> 
> The addition of `__noreturn__` to the `__attribute__` seems unrelated
> to the stated purpose of this patch. As such, it typically would be
> placed in its own patch. If it really is too minor for a separate
> patch, mentioning it in the commit message as a "While at it..." would
> be helpful.

Acknowledged. It is indeed a bit of a nothing change that doesn’t really do much on its own, but when paired with the port variable reorder could potentially make a “minor code cleanup” commit.

>> +       va_start(args, allocator);
>> +       while ((key = va_arg(args, const void *)) != NULL) {
>> +               const void *value;
>> +               value = va_arg(args, const void *);
>> +               if (value)
>> +                       CFDictionarySetValue(result, key, value);
>> +       }
>> +       va_end(args);
> 
> A couple related comments...
> 
> If va_arg() ever returns NULL for `value`, the next iteration of the
> loop will call va_arg() again, but calling va_arg() again after it has
> already returned NULL is likely undefined behavior. At minimum, I
> would have expected this to be written as:
> 
> while (...) {
>     ...
>     if (!value)
>         break;
>     CFDictionarySetValue(...);
> }
> 
> However, isn't it a programmer error if va_arg() returns NULL for
> `value`? If so, I'd think we'd want to scream loudly about that rather
> than silently ignoring it. So, perhaps something like this:
> 
> while (...) {
>     ...
>     if (!value) {
>         fprintf(stderr, "BUG: ...");
>         abort();
>     }
>     CFDictionarySetValue(...);
> }
> 
> Or, perhaps just call the existing die() function in this file with a
> suitable "BUG ..." message.
> 

In this case it’s by design to accept and check for NULL values as it greatly simplifies the code. Inputs to the credential helpers have various optional fields, such as port and path. It is programmer error to pass NULL to the SecItem API (runtime crash) so in order to simplify having to check each individual field in all of the callers (and probably ditch varargs since you can’t really do dynamic varargs), I check the value here instead. That means you can do something like:

 create_dictionary(kCFAllocatorDefault,
     kSecAttrServer, host,
     kSecAttrPath, path, \
     kSecAttrPort, port,
     NULL)

And it will only include the key-value pairs that have non-NULL values.

It would indeed be programmer error to not pass key-value pairs, though it is equally programmer error to not have a terminating NULL.

>> +       username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
>> +       if (username_buf)
>> +       {
>> +               write_item("username", username_buf, strlen(username_buf));
>>             return;
>> +       }
> 
> According to the documentation for CFStringGetCStringPtr(), the
> returned C-string is not newly-allocated, so the caller does not have
> to free it. Therefore, can `username_buf` be declared `const char *`
> rather than `char *` to make it clear to readers that nothing is being
> leaked here? Same comment about the `(char *)` cast.
> 
>> +       /* If we can't get a CString pointer then
>> +        * we need to allocate our own buffer */
> 
> Style:
> 
> /*
>  * Multi-line comments
>  * are formatted like this.
>  */
> 
>> +       buffer_len = CFStringGetMaximumSizeForEncoding(
>> +                       CFStringGetLength(account_ref), ENCODING) + 1;
>> +       username_buf = xmalloc(buffer_len);
>> +       if (CFStringGetCString(account_ref,
>> +                               username_buf,
>> +                               buffer_len,
>> +                               ENCODING)) {
>> +               write_item("username", username_buf, buffer_len - 1);
>> +       }
>> +       free(username_buf);
> 
> Okay, this explains why `username_buf` is declared `char *` rather
> than `const char *`. Typically, when we have a situation in which a
> value may or may not need freeing, we use a `to_free` variable like
> this:
> 
> const char *username_buf;
> char *to_free = NULL;
> ...
> username_buf = (const char *)CFStringGetCStringPtr(...);
> if (username_buf) {
>     ...
>     return;
> }
> ...
> username_buf = to_free = xmalloc(buffer_len);
> if (CFStringGetCString(...))
>     ...
> free(to_free);
> 
> But that may be overkill for this simple case, and what you have here
> may be "good enough" for anyone already familiar with the API and who
> knows that the `return` after CFStringGetCStringPtr() isn't leaking.

Would it make sense to just have a comment paired with the CFStringGetCStringPtr return explaining why it doesn’t need to be freed there? I’m OK with the to_free variable however if that’s clearer. Idea in my mind was pairing it based on `xmalloc` but I can see why pairing based on variable is clearer.

Re: [PATCH 1/4] osxkeychain: replace deprecated SecKeychain API

[Eric Sunshine]

On Sun, Feb 18, 2024 at 9:48 AM Bo Anderson <mail@boanderson.me> wrote:
> > On 18 Feb 2024, at 06:08, Eric Sunshine <sunshine@sunshineco.com> wrote:
> >> +    va_start(args, allocator);
> >> +    while ((key = va_arg(args, const void *)) != NULL) {
> >> +        const void *value;
> >> +        value = va_arg(args, const void *);
> >> +        if (value)
> >> +            CFDictionarySetValue(result, key, value);
> >> +    }
> >> +    va_end(args);
> >
> > However, isn't it a programmer error if va_arg() returns NULL for
> > `value`? If so, I'd think we'd want to scream loudly about that rather
> > than silently ignoring it. So, perhaps something like this: [...]
>
> In this case it’s by design to accept and check for NULL values as
> it greatly simplifies the code. Inputs to the credential helpers
> have various optional fields, such as port and path. It is
> programmer error to pass NULL to the SecItem API (runtime crash) so
> in order to simplify having to check each individual field in all of
> the callers (and probably ditch varargs since you can’t really do
> dynamic varargs), I check the value here instead. That means you can
> do something like:
>
> create_dictionary(kCFAllocatorDefault,
>   kSecAttrServer, host,
>   kSecAttrPath, path, \
>   kSecAttrPort, port,
>   NULL)
>
> And it will only include the key-value pairs that have non-NULL
> values.
>
> It would indeed be programmer error to  not pass key-value pairs,
> though it is equally programmer  error to  not have a terminating
> NULL.

Okay. I had thought that this check was merely protecting against
programmer error, but the described use-case to avoid passing NULL to
SecItem API makes perfect sense. It might be helpful to future readers
to explain this either as a function-level comment (explaining how to
call the function) or as an in-code comment.

> >> +    username_buf = (char *)CFStringGetCStringPtr(account_ref, ENCODING);
> >> +    if (username_buf)
> >> +    {
> >> +        write_item("username", username_buf, strlen(username_buf));
> >>       return;
> >> +    }
> >
> > According to the documentation for CFStringGetCStringPtr(), the
> > returned C-string is not newly-allocated, so the caller does not have
> > to free it. Therefore, can `username_buf` be declared `const char *`
> > rather than `char *` to make it clear to readers that nothing is being
> > leaked here? Same comment about the `(char *)` cast.
> >
> >> +    buffer_len = CFStringGetMaximumSizeForEncoding(
> >> +            CFStringGetLength(account_ref), ENCODING) + 1;
> >> +    username_buf = xmalloc(buffer_len);
> >> +    if (CFStringGetCString(account_ref,
> >> +                username_buf,
> >> +                buffer_len,
> >> +                ENCODING)) {
> >> +        write_item("username", username_buf, buffer_len - 1);
> >> +    }
> >> +    free(username_buf);
> >
> > Okay, this explains why `username_buf` is declared `char *` rather
> > than `const char *`. Typically, when we have a situation in which a
> > value may or may not need freeing, we use a `to_free` variable like
> > this: [...]
> >
> > But that may be overkill for this simple case, and what you have here
> > may be "good enough" for anyone already familiar with the API and who
> > knows that the `return` after CFStringGetCStringPtr() isn't leaking.
>
> Would it make sense to just have a comment paired with the
> CFStringGetCStringPtr return explaining why it doesn’t need to be
> freed there? I’m OK with the to_free variable however if that’s
> clearer. Idea in my mind was pairing it based on `xmalloc` but I can
> see why pairing based on variable is clearer.

Most likely, anyone working on this code is already familiar with the
CoreFoundation API, thus would understand implicitly that this isn't
leaking. But, yes, a simple comment should be plenty sufficient for
everyone else if you are re-rolling anyhow.

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[M Hickford]

> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. As it stood, osxkeychain failed seven tests
> in the external credential helper test suite:
> 
> not ok 8 - helper (osxkeychain) overwrites on store
> not ok 9 - helper (osxkeychain) can forget host
> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
> not ok 15 - helper (osxkeychain) erases all matching credentials
> not ok 18 - helper (osxkeychain) gets password_expiry_utc
> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
>
> After this set of patches, osxkeychain passes all tests in the external
credential helper test suite.

Great work!

Could these tests run as part of macOS CI?

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Bo Anderson]

> On 18 Feb 2024, at 20:40, M Hickford <mirth.hickford@gmail.com> wrote:
> 
>> git-credential-osxkeychain has largely fallen behind other external
>> credential helpers in the features it supports, and hasn't received any
>> functional changes since 2013. As it stood, osxkeychain failed seven tests
>> in the external credential helper test suite:
>> 
>> not ok 8 - helper (osxkeychain) overwrites on store
>> not ok 9 - helper (osxkeychain) can forget host
>> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
>> not ok 15 - helper (osxkeychain) erases all matching credentials
>> not ok 18 - helper (osxkeychain) gets password_expiry_utc
>> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
>> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
>> 
>> After this set of patches, osxkeychain passes all tests in the external
> credential helper test suite.
> 
> Great work!
> 
> Could these tests run as part of macOS CI?

Do we do so for any of the other external credential helpers?

It definitely makes sense in principle. Though the concern perhaps will be that any new features added to the credential helpers and thus its test suite would need adding to each credential helper simultaneously to avoid failing CI. Ideally we would do exactly that, though that requires knowledge on each of the keystore APIs used in each of the credential helpers.

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[M Hickford]

On Sun, 18 Feb 2024 at 23:24, Bo Anderson <mail@boanderson.me> wrote:
>
> > On 18 Feb 2024, at 20:40, M Hickford <mirth.hickford@gmail.com> wrote:
> >
> >
> > Could these tests run as part of macOS CI?
>
> Do we do so for any of the other external credential helpers?
>

We don't.

> It definitely makes sense in principle. Though the concern perhaps will be that any new features added to the credential helpers and thus its test suite would need adding to each credential helper simultaneously to avoid failing CI. Ideally we would do exactly that, though that requires knowledge on each of the keystore APIs used in each of the credential helpers.

Good point.

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Jeff King]

On Mon, Mar 04, 2024 at 08:00:00AM +0000, M Hickford wrote:

> > It definitely makes sense in principle. Though the concern perhaps
> > will be that any new features added to the credential helpers and
> > thus its test suite would need adding to each credential helper
> > simultaneously to avoid failing CI. Ideally we would do exactly
> > that, though that requires knowledge on each of the keystore APIs
> > used in each of the credential helpers.
> 
> Good point.

I think we suffer from that somewhat already. You cannot run t0303
successfully against credential-store anymore, as of 0ce02e2fec
(credential/libsecret: store new attributes, 2023-06-16).

There is some prior art in the GIT_TEST_CREDENTIAL_HELPER_TIMEOUT
variable, as time is not a concept to every helper (like store, for
example). Other new tests like the password-expiry and oauth features
could be gated on similar variables. That would help non-CI users
testing helpers manually, and then CI jobs could set the appropriate
switches for each helper that they cover.

All that said, I'd be surprised if testing osxkeychain in the CI
environment worked. Back when I worked on it in 2011, I found that I had
to actually run the tests in a local terminal; even a remote ssh login
could not access the keychain. It's possible that things have changed
since then, though, or perhaps I was imply ignorant of how to configure
things correctly.

-Peff

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[M Hickford]

> From: "Bo Anderson via GitGitGadget" <gitgitgadget@gmail.com>
> To: git@vger.kernel.org
> Cc: Bo Anderson <mail@boanderson.me>
> Subject: [PATCH 0/4] osxkeychain: bring in line with other credential helpers
> Date: Sat, 17 Feb 2024 23:34:52 +0000	[thread overview]
> Message-ID: <pull.1667.git.1708212896.gitgitgadget@gmail.com> (raw)
> 
> git-credential-osxkeychain has largely fallen behind other external
> credential helpers in the features it supports, and hasn't received any
> functional changes since 2013. As it stood, osxkeychain failed seven tests
> in the external credential helper test suite:
> 
> not ok 8 - helper (osxkeychain) overwrites on store
> not ok 9 - helper (osxkeychain) can forget host
> not ok 11 - helper (osxkeychain) does not erase a password distinct from input
> not ok 15 - helper (osxkeychain) erases all matching credentials
> not ok 18 - helper (osxkeychain) gets password_expiry_utc
> not ok 19 - helper (osxkeychain) overwrites when password_expiry_utc changes
> not ok 21 - helper (osxkeychain) gets oauth_refresh_token
> 
> 
> osxkeychain also made use of macOS APIs that had been deprecated since 2014.
> Replacement API was able to be used without regressing the minimum supported
> macOS established in 5747c8072b (contrib/credential: avoid fixed-size buffer
> in osxkeychain, 2023-05-01).
> 
> After this set of patches, osxkeychain passes all tests in the external
> credential helper test suite.
> 
> Bo Anderson (4):
>   osxkeychain: replace deprecated SecKeychain API
>   osxkeychain: erase all matching credentials
>   osxkeychain: erase matching passwords only
>   osxkeychain: store new attributes
> 
>  contrib/credential/osxkeychain/Makefile       |   3 +-
>  .../osxkeychain/git-credential-osxkeychain.c  | 376 ++++++++++++++----
>  2 files changed, 310 insertions(+), 69 deletions(-)
> 
> 
> base-commit: 3e0d3cd5c7def4808247caf168e17f2bbf47892b
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1667%2FBo98%2Fosxkeychain-update-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1667/Bo98/osxkeychain-update-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1667
> -- 
> gitgitgadget

Hi. Is this patch ready to cook in seen?

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Junio C Hamano]

M Hickford <mirth.hickford@gmail.com> writes:

>> From: "Bo Anderson via GitGitGadget" <gitgitgadget@gmail.com>
>> To: git@vger.kernel.org
>> Cc: Bo Anderson <mail@boanderson.me>
>> Subject: [PATCH 0/4] osxkeychain: bring in line with other credential helpers
>> Date: Sat, 17 Feb 2024 23:34:52 +0000	[thread overview]
>> Message-ID: <pull.1667.git.1708212896.gitgitgadget@gmail.com> (raw)
>>  ...
>> git-credential-osxkeychain has largely fallen behind other external
> Hi. Is this patch ready to cook in seen?

A better nudge would be for somebody who uses macOS to resend the
patches with their "Tested-by:" trailers added ;-).

Thanks.

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Robert Coup]

Hi all,

> All that said, I'd be surprised if testing osxkeychain in the CI
> environment worked. Back when I worked on it in 2011, I found that I had
> to actually run the tests in a local terminal; even a remote ssh login
> could not access the keychain. It's possible that things have changed
> since then, though, or perhaps I was imply ignorant of how to configure
> things correctly.

I have gotten keychain working in Github Actions before: there's some
helpers for it, but you can also basically do it manually via the
steps from [1]. Basically anyone who needs to do Apple code-signing in
CI has to make it work.

@Bo, how are you actually testing this manually? Following these steps:

$ make
$ (cd contrib/credential/osxkeychain && make)
$ ln -s contrib/credential/osxkeychain/git-credential-osxkeychain .
$ cd t
$ make GIT_TEST_CREDENTIAL_HELPER=osxkeychain t0303-credential-external.sh

I get 'A keychain cannot be found to store "store-user".' in a popup
dialog when #2 runs; then similar for other tests in 0303. For #14 I
get a slight alternative with "A keychain cannot be found". There's a
"Reset To Defaults" button, but that wipes everything. AFAIK I have a
relatively normal setup, with a login keychain as default. macOS
14.3.1; arm64.

$ security list-keychains
    "/Users/rc/Library/Keychains/login.keychain-db"
    "/Library/Keychains/System.keychain"
$ security default-keychain
    "/Users/rc/Library/Keychains/login.keychain-db"
$ security unlock-keychain
password to unlock default: ...

I don't see any settings or code for setting which keychain the
credential helper uses, so I guess it's the default one?

Cheers,

Rob :)

[1] https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Bo Anderson]

The test script does not interact well with the env filtering. This was the case before this change too.

To interact with your default keychain, you will need:

GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”

This is because the default macOS user keychain is local to your home directory - that’s why it’s giving errors about not finding any.

Bo

> On 2 Apr 2024, at 14:21, Robert Coup <robert.coup@koordinates.com> wrote:
> 
> Hi all,
> 
>> All that said, I'd be surprised if testing osxkeychain in the CI
>> environment worked. Back when I worked on it in 2011, I found that I had
>> to actually run the tests in a local terminal; even a remote ssh login
>> could not access the keychain. It's possible that things have changed
>> since then, though, or perhaps I was imply ignorant of how to configure
>> things correctly.
> 
> I have gotten keychain working in Github Actions before: there's some
> helpers for it, but you can also basically do it manually via the
> steps from [1]. Basically anyone who needs to do Apple code-signing in
> CI has to make it work.
> 
> @Bo, how are you actually testing this manually? Following these steps:
> 
> $ make
> $ (cd contrib/credential/osxkeychain && make)
> $ ln -s contrib/credential/osxkeychain/git-credential-osxkeychain .
> $ cd t
> $ make GIT_TEST_CREDENTIAL_HELPER=osxkeychain t0303-credential-external.sh
> 
> I get 'A keychain cannot be found to store "store-user".' in a popup
> dialog when #2 runs; then similar for other tests in 0303. For #14 I
> get a slight alternative with "A keychain cannot be found". There's a
> "Reset To Defaults" button, but that wipes everything. AFAIK I have a
> relatively normal setup, with a login keychain as default. macOS
> 14.3.1; arm64.
> 
> $ security list-keychains
>    "/Users/rc/Library/Keychains/login.keychain-db"
>    "/Library/Keychains/System.keychain"
> $ security default-keychain
>    "/Users/rc/Library/Keychains/login.keychain-db"
> $ security unlock-keychain
> password to unlock default: ...
> 
> I don't see any settings or code for setting which keychain the
> credential helper uses, so I guess it's the default one?
> 
> Cheers,
> 
> Rob :)
> 
> [1] https://docs.github.com/en/actions/deployment/deploying-xcode-applications/installing-an-apple-certificate-on-macos-runners-for-xcode-development

Re: [PATCH 0/4] osxkeychain: bring in line with other credential helpers

[Robert Coup]

Hi Bo,

On Tue, 2 Apr 2024 at 14:53, Bo Anderson <mail@boanderson.me> wrote:
>
> The test script does not interact well with the env filtering. This was the case before this change too.

I guess without writing a helper-specific test or having some
per-helper-setup thing it's a bit tricky.

> To interact with your default keychain, you will need:
>
> GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”
>
> This is because the default macOS user keychain is local to your home directory - that’s why it’s giving errors about not finding any.

And with that, the tests all pass :-) Comparing with master where 7/21 failed.

Tested-by: Robert Coup <robert.coup@koordinates.com>

Could we document that setup step somewhere? I guess the simplest is
probably just to put it in the header of
t/t0303-credential-external.sh; maybe along the lines of the patch
below.

Rob :)


diff --git a/t/t0303-credential-external.sh b/t/t0303-credential-external.sh
index 095574bfc6..e4e693b233 100755
--- a/t/t0303-credential-external.sh
+++ b/t/t0303-credential-external.sh
@@ -27,6 +27,13 @@ timeout, you can test that feature with:
 If your helper requires additional setup before the tests are started,
 you can set GIT_TEST_CREDENTIAL_HELPER_SETUP to a sequence of shell
 commands.
+
+- osxkeychain:
+
+  Because the default macOS user keychain is local to your home
+  directory, you will need:
+
+    GIT_TEST_CREDENTIAL_HELPER_SETUP="export HOME=$HOME”
 '

 . ./test-lib.sh