git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: "Eric Sunshine" <sunshine@sunshineco.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Eric DeCosta" <edecosta@mathworks.com>,
	"Eric DeCosta" <edecosta@mathworks.com>
Subject: [PATCH v2 04/12] fsmonitor: deal with synthetic firmlinks on macOS
Date: Fri, 14 Oct 2022 21:45:36 +0000	[thread overview]
Message-ID: <d8f20032d6b9e64ecd7e8cdd191cbbc516287dc8.1665783944.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1352.v2.git.git.1665783944.gitgitgadget@gmail.com>

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     | 14 +++-
 compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  3 +
 fsmonitor-path-utils.h                   | 36 +++++++++-
 6 files changed, 167 insertions(+), 3 deletions(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 6e417846d8b..6f30a4f93a7 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,11 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	strbuf_init(&state.alias.alias, 0);
+	strbuf_init(&state.alias.points_to, 0);
+	if ((err = fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)))
+		goto done;
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
@@ -1391,6 +1397,8 @@ done:
 	strbuf_release(&state.path_gitdir_watch);
 	strbuf_release(&state.path_cookie_prefix);
 	strbuf_release(&state.path_ipc);
+	strbuf_release(&state.alias.alias);
+	strbuf_release(&state.alias.points_to);
 
 	return err;
 }
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..daeee4e465c 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -198,8 +199,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 	struct string_list cookie_list = STRING_LIST_INIT_DUP;
 	const char *path_k;
 	const char *slash;
-	int k;
+	char *resolved = NULL;
 	struct strbuf tmp = STRBUF_INIT;
+	int k;
 
 	/*
 	 * Build a list of all filesystem changes into a private/local
@@ -209,7 +211,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		free(resolved);
+		resolved = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (resolved)
+			path_k = resolved;
+		else
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +245,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
@@ -360,12 +368,14 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		}
 	}
 
+	free(resolved);
 	fsmonitor_publish(state, batch, &cookie_list);
 	string_list_clear(&cookie_list, 0);
 	strbuf_release(&tmp);
 	return;
 
 force_shutdown:
+	free(resolved);
 	fsmonitor_batch__free_list(batch);
 	string_list_clear(&cookie_list, 0);
 
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index d46d7f13538..ce5a8febe09 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -41,3 +44,92 @@ int fsmonitor__is_fs_remote(const char *path)
 
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR *dir;
+	int retval = -1;
+	const char *const root = "/";
+	struct stat st;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to = STRBUF_INIT;
+
+	dir = opendir(root);
+	if (!dir)
+		return error_errno(_("opendir('%s') failed"), root);
+
+	strbuf_init(&alias, 256);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addf(&alias, "%s%s", root, de->d_name);
+
+		if (lstat(alias.buf, &st) < 0) {
+			error_errno(_("lstat('%s') failed"), alias.buf);
+			goto done;
+		}
+
+		if (!S_ISLNK(st.st_mode))
+			continue;
+
+		if (strbuf_readlink(&points_to, alias.buf, st.st_size) < 0) {
+			error_errno(_("strbuf_readlink('%s') failed"), alias.buf);
+			goto done;
+		}
+
+		if (!strncmp(points_to.buf, path, points_to.len) &&
+			(path[points_to.len] == '/')) {
+			strbuf_addbuf(&info->alias, &alias);
+			strbuf_addbuf(&info->points_to, &points_to);
+			trace_printf_key(&trace_fsmonitor,
+				"Found alias for '%s' : '%s' -> '%s'",
+				path, info->alias.buf, info->points_to.buf);
+			retval = 0;
+			goto done;
+		}
+	}
+	retval = 0; /* no alias */
+
+done:
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	if (closedir(dir) < 0)
+		return error_errno(_("closedir('%s') failed"), root);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	if (!info->alias.len)
+		return NULL;
+
+	if ((!strncmp(info->alias.buf, path, info->alias.len))
+		&& path[info->alias.len] == '/') {
+		struct strbuf tmp = STRBUF_INIT;
+		const char *remainder = path + info->alias.len;
+
+		strbuf_addbuf(&tmp, &info->points_to);
+		strbuf_add(&tmp, remainder, strlen(remainder));
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index a90b8f7925b..0d95bbb416f 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..e24838f9a86 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index 41edf5b934f..5bfdfb81c14 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,13 +1,21 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	struct strbuf alias;
+	struct strbuf points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
 };
 
 /*
- * Get some basic filesystem informtion for the given path
+ * Get some basic filesystem information for the given path
  *
  * The caller owns the storage that is occupied by fs_info and
  * is responsible for releasing it.
@@ -23,4 +31,30 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by info.alias and
+ * info.points_to and is responsible for releasing it.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
 #endif
-- 
gitgitgadget


  parent reply	other threads:[~2022-10-14 21:46 UTC|newest]

Thread overview: 89+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-09 14:37 [PATCH 00/12] fsmonitor: Implement fsmonitor for Linux Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 01/12] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
2022-10-18 12:29   ` Ævar Arnfjörð Bjarmason
2022-10-09 14:37 ` [PATCH 02/12] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 03/12] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 04/12] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 05/12] fsmonitor: check for compatability before communicating with fsmonitor Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 06/12] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 07/12] fsmonitor: prepare to share code between Mac OS and Linux Eric DeCosta via GitGitGadget
2022-10-09 22:36   ` Junio C Hamano
2022-10-10 16:23   ` Jeff Hostetler
2022-10-09 14:37 ` [PATCH 08/12] fsmonitor: determine if filesystem is local or remote Eric DeCosta via GitGitGadget
2022-10-10 10:04   ` Ævar Arnfjörð Bjarmason
2022-10-14 19:38     ` Eric DeCosta
2022-10-09 14:37 ` [PATCH 09/12] fsmonitor: implement filesystem change listener for Linux Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 10/12] fsmonitor: enable fsmonitor " Eric DeCosta via GitGitGadget
2022-10-09 14:37 ` [PATCH 11/12] fsmonitor: test updates Eric DeCosta via GitGitGadget
2022-10-18 12:42   ` Ævar Arnfjörð Bjarmason
2022-10-09 14:37 ` [PATCH 12/12] fsmonitor: update doc for Linux Eric DeCosta via GitGitGadget
2022-10-18 12:43   ` Ævar Arnfjörð Bjarmason
2022-10-09 22:24 ` [PATCH 00/12] fsmonitor: Implement fsmonitor " Junio C Hamano
2022-10-10  0:19   ` Eric Sunshine
2022-10-10 21:08 ` Junio C Hamano
2022-10-14 21:45 ` [PATCH v2 " Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 01/12] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 02/12] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
2022-10-18 12:12     ` Ævar Arnfjörð Bjarmason
2022-10-14 21:45   ` [PATCH v2 03/12] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` Eric DeCosta via GitGitGadget [this message]
2022-10-14 21:45   ` [PATCH v2 05/12] fsmonitor: check for compatability before communicating with fsmonitor Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 06/12] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 07/12] fsmonitor: prepare to share code between Mac OS and Linux Eric DeCosta via GitGitGadget
2022-10-14 23:46     ` Junio C Hamano
2022-10-17 21:30       ` Eric DeCosta
2022-10-18  6:31         ` Junio C Hamano
2022-10-14 21:45   ` [PATCH v2 08/12] fsmonitor: determine if filesystem is local or remote Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 09/12] fsmonitor: implement filesystem change listener for Linux Eric DeCosta via GitGitGadget
2022-10-18 11:59     ` Ævar Arnfjörð Bjarmason
2022-10-14 21:45   ` [PATCH v2 10/12] fsmonitor: enable fsmonitor " Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 11/12] fsmonitor: test updates Eric DeCosta via GitGitGadget
2022-10-14 21:45   ` [PATCH v2 12/12] fsmonitor: update doc for Linux Eric DeCosta via GitGitGadget
2022-10-14 23:32   ` [PATCH v2 00/12] fsmonitor: Implement fsmonitor " Junio C Hamano
2022-10-17 21:32     ` Eric DeCosta
2022-10-17 22:22       ` Junio C Hamano
2022-10-18 14:02       ` Johannes Schindelin
2022-10-17 22:14   ` Glen Choo
2022-10-18  4:17     ` Junio C Hamano
2022-10-18 17:03       ` Glen Choo
2022-10-18 17:11         ` Junio C Hamano
2022-10-19  1:19         ` Ævar Arnfjörð Bjarmason
2022-10-19  2:28           ` Eric Sunshine
2022-10-19 16:58             ` Junio C Hamano
2022-10-19 19:11             ` Ævar Arnfjörð Bjarmason
2022-10-19 20:14               ` Eric Sunshine
2022-10-19  1:04       ` Ævar Arnfjörð Bjarmason
2022-10-19 16:33         ` Junio C Hamano
2022-10-20 16:13       ` Junio C Hamano
2022-10-20 16:37         ` Eric Sunshine
2022-10-20 17:01           ` Junio C Hamano
2022-10-20 17:54             ` Eric Sunshine
2022-10-20 15:43     ` Junio C Hamano
2022-10-20 22:01       ` Junio C Hamano
2022-11-16 23:23   ` [PATCH v3 0/6] " Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 1/6] fsmonitor: prepare to share code between Mac OS and Linux Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 2/6] fsmonitor: determine if filesystem is local or remote Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 3/6] fsmonitor: implement filesystem change listener for Linux Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 4/6] fsmonitor: enable fsmonitor " Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 5/6] fsmonitor: test updates Eric DeCosta via GitGitGadget
2022-11-16 23:23     ` [PATCH v3 6/6] fsmonitor: update doc for Linux Eric DeCosta via GitGitGadget
2022-11-16 23:27     ` [PATCH v3 0/6] fsmonitor: Implement fsmonitor " Taylor Blau
2022-11-23 19:00     ` [PATCH v4 " Eric DeCosta via GitGitGadget
2022-11-23 19:00       ` [PATCH v4 1/6] fsmonitor: prepare to share code between Mac OS and Linux Eric DeCosta via GitGitGadget
2022-11-23 19:00       ` [PATCH v4 2/6] fsmonitor: determine if filesystem is local or remote Eric DeCosta via GitGitGadget
2022-11-25  7:31         ` Junio C Hamano
2022-12-12 10:24         ` Ævar Arnfjörð Bjarmason
2022-11-23 19:00       ` [PATCH v4 3/6] fsmonitor: implement filesystem change listener for Linux Eric DeCosta via GitGitGadget
2022-12-12 10:42         ` Ævar Arnfjörð Bjarmason
2022-11-23 19:00       ` [PATCH v4 4/6] fsmonitor: enable fsmonitor " Eric DeCosta via GitGitGadget
2022-11-23 19:00       ` [PATCH v4 5/6] fsmonitor: test updates Eric DeCosta via GitGitGadget
2022-11-23 19:00       ` [PATCH v4 6/6] fsmonitor: update doc for Linux Eric DeCosta via GitGitGadget
2022-12-12 21:57       ` [PATCH v5 0/6] fsmonitor: Implement fsmonitor " Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 1/6] fsmonitor: prepare to share code between Mac OS and Linux Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 2/6] fsmonitor: determine if filesystem is local or remote Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 3/6] fsmonitor: implement filesystem change listener for Linux Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 4/6] fsmonitor: enable fsmonitor " Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 5/6] fsmonitor: test updates Eric DeCosta via GitGitGadget
2022-12-12 21:58         ` [PATCH v5 6/6] fsmonitor: update doc for Linux Eric DeCosta via GitGitGadget
2023-04-12  5:42         ` [PATCH v5 0/6] fsmonitor: Implement fsmonitor " Junio C Hamano
2022-10-18 12:47 ` [PATCH 00/12] " Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d8f20032d6b9e64ecd7e8cdd191cbbc516287dc8.1665783944.git.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=avarab@gmail.com \
    --cc=edecosta@mathworks.com \
    --cc=git@vger.kernel.org \
    --cc=sunshine@sunshineco.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).