git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
@ 2022-08-18 20:48 Eric DeCosta via GitGitGadget
  2022-08-18 21:35 ` Junio C Hamano
                   ` (6 more replies)
  0 siblings, 7 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-18 20:48 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Though perhaps not common, there are uses cases where users have large,
network-mounted repos. Having the ability to run fsmonitor against
network paths would benefit those users.

As a first step towards enabling fsmonitor to work against
network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
was introduced for Windows. Setting this option to true will override
the default behavior (erroring-out) when a network-mounted repo is
detected by fsmonitor. In order for macOS to have parity with Windows,
the same option is now introduced for macOS.

The the added wrinkle being that the Unix domain socket (UDS) file
used for IPC cannot be created in a network location; instead the
temporary directory is used.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
    fsmonitor: option to allow fsmonitor to run against network-mounted
    repos

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

 compat/fsmonitor/fsm-settings-darwin.c | 77 ++++++++++++++++++++++----
 fsmonitor-ipc.c                        | 47 +++++++++++++++-
 fsmonitor-ipc.h                        |  6 ++
 3 files changed, 117 insertions(+), 13 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..9e2ea3b90cc 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -2,10 +2,28 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-ipc.h"
 #include "fsmonitor.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
+/*
+ * Check if monitoring remote working directories is allowed.
+ *
+ * By default, monitoring remote working directories is
+ * disabled.  Users may override this behavior in enviroments where
+ * they have proper support.
+ */
+static int check_config_allowremote(struct repository *r)
+{
+	int allow;
+
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		return allow;
+
+	return -1; /* fsmonitor.allowremote not set */
+}
+
 /*
  * [1] Remote working directories are problematic for FSMonitor.
  *
@@ -27,24 +45,22 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
+ * So (for now at least), mark remote working directories as
+ * incompatible by default.
+ *
  * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
+ * IPC in the temporary directory.  If the temporary directory is
+ * remote, then the socket will be created on the remote file system.
+ * This can fail if the remote file system does not support UDS file
+ * types (e.g. smbfs to a Windows server) or if the remote kernel does
+ * not allow a non-local process to bind() the socket.
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
+ * Therefore remote UDS locations are marked as incompatible.
  *
  *
  * [2] FAT32 and NTFS working directories are problematic too.
  *
- * The builtin FSMonitor uses a Unix domain socket in the .git
+ * The builtin FSMonitor uses a Unix domain socket in the temporary
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
@@ -65,6 +81,39 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)) {
+		switch (check_config_allowremote(r)) {
+		case 0: /* config overrides and disables */
+			return FSMONITOR_REASON_REMOTE;
+		case 1: /* config overrides and enables */
+			return FSMONITOR_REASON_OK;
+		default:
+			break; /* config has no opinion */
+		}
+
+		return FSMONITOR_REASON_REMOTE;
+	}
+
+	return FSMONITOR_REASON_OK;
+}
+
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	const char *path = fsmonitor_ipc__get_path();
+
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +134,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..6e9b40a03d5 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -4,6 +4,7 @@
 #include "fsmonitor-ipc.h"
 #include "run-command.h"
 #include "strbuf.h"
+#include "tempfile.h"
 #include "trace2.h"
 
 #ifndef HAVE_FSMONITOR_DAEMON_BACKEND
@@ -47,7 +48,51 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+GIT_PATH_FUNC(fsmonitor_ipc__get_pathfile, "fsmonitor--daemon.ipc")
+
+static char *gen_ipc_file(void)
+{
+	char *retval = NULL;
+	struct tempfile *ipc;
+
+	const char *ipc_file = fsmonitor_ipc__get_pathfile();
+	FILE *fp = fopen(ipc_file, "w");
+
+	if (!fp)
+		die_errno("error opening '%s'", ipc_file);
+	ipc = mks_tempfile_t("fsmonitor_ipc_XXXXXX");
+	strbuf_write(&ipc->filename, fp);
+	fclose(fp);
+	retval = strbuf_detach(&ipc->filename, NULL);
+	strbuf_release(&ipc->filename);
+	return retval;
+}
+
+const char *fsmonitor_ipc__get_path(void)
+{
+	char *retval = NULL;
+	struct strbuf sb = STRBUF_INIT;
+
+	const char *ipc_file = fsmonitor_ipc__get_pathfile();
+	FILE *fp = fopen(ipc_file, "r");
+
+	if (!fp) {
+		return gen_ipc_file();
+	} else {
+		strbuf_read(&sb, fileno(fp), 0);
+		fclose(fp);
+		fp = fopen(sb.buf, "r");
+		if (!fp) { /* generate new file */
+			if (unlink(ipc_file) < 0)
+				die_errno("could not remove '%s'", ipc_file);
+			return gen_ipc_file();
+		}
+		fclose(fp);
+		retval = strbuf_detach(&sb, NULL);
+		strbuf_release(&sb);
+		return retval;
+	}
+}
 
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..63277dea39e 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the file that contains the pathname to the
+ * IPC named pipe or Unix domain socket.
+ */
+const char *fsmonitor_ipc__get_pathfile(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.

base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
-- 
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the file that contains the pathname to the
+ * IPC named pipe or Unix domain socket.
+ */
+const char *fsmonitor_ipc__get_pathfile(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.

base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
@ 2022-08-18 21:35 ` Junio C Hamano
  2022-08-18 21:38   ` Junio C Hamano
  2022-08-19 10:05 ` Johannes Schindelin
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-08-18 21:35 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> Though perhaps not common, there are uses cases where users have large,
> network-mounted repos. Having the ability to run fsmonitor against
> network paths would benefit those users.
>
> As a first step towards enabling fsmonitor to work ...

Didn't we already see the first step and got it reviewed?  I think
I've already merged the last round to 'next'.

Puzzled...

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 21:35 ` Junio C Hamano
@ 2022-08-18 21:38   ` Junio C Hamano
  0 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-08-18 21:38 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

Junio C Hamano <gitster@pobox.com> writes:

> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
>
>> From: Eric DeCosta <edecosta@mathworks.com>
>>
>> Though perhaps not common, there are uses cases where users have large,
>> network-mounted repos. Having the ability to run fsmonitor against
>> network paths would benefit those users.
>>
>> As a first step towards enabling fsmonitor to work ...
>
> Didn't we already see the first step and got it reviewed?  I think
> I've already merged the last round to 'next'.
>
> Puzzled...

Ah, OK, so this is not exactly the "first step", but builds on top
of the previous one?  At least the patch needs a better title to
make it distinguishable from the other one.

Will queue to wait for others to review.

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-08-18 21:35 ` Junio C Hamano
@ 2022-08-19 10:05 ` Johannes Schindelin
  2022-08-19 16:50 ` Jeff Hostetler
                   ` (4 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Johannes Schindelin @ 2022-08-19 10:05 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta, Eric DeCosta

Hi Eric,

On Thu, 18 Aug 2022, Eric DeCosta via GitGitGadget wrote:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> Though perhaps not common, there are uses cases where users have large,
> network-mounted repos. Having the ability to run fsmonitor against
> network paths would benefit those users.
>
> As a first step towards enabling fsmonitor to work against
> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> was introduced for Windows.

If you start the commit message along the following lines, it might be
easier/quicker to grok the context for the keen reader:

	In 85dc0da6dcf (fsmonitor: option to allow fsmonitor to run against
	network-mounted repos, 2022-08-11), the Windows backend of the
	FSMonitor learned to allow running on network drives, via the
	`fsmonitor.allowRemote` config setting.

> Setting this option to true will override the default behavior
> (erroring-out) when a network-mounted repo is detected by fsmonitor. In
> order for macOS to have parity with Windows, the same option is now
> introduced for macOS.
>
> The the added wrinkle being that the Unix domain socket (UDS) file
> used for IPC cannot be created in a network location; instead the
> temporary directory is used.

Thank you very much for this note, after a cursory read I expected that
part of the code to be a left-over from some "We know better than the
user" type of automatic default, and this paragraph definitely helped me
overcome that expectation.

>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
>     fsmonitor: option to allow fsmonitor to run against network-mounted
>     repos
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/1326
>
>  compat/fsmonitor/fsm-settings-darwin.c | 77 ++++++++++++++++++++++----
>  fsmonitor-ipc.c                        | 47 +++++++++++++++-
>  fsmonitor-ipc.h                        |  6 ++
>  3 files changed, 117 insertions(+), 13 deletions(-)

I am somewhat puzzled that this has no corresponding change to
`Documentation/`.

And now I realize that this was the case also for the patch adding
`fsmonitor.allowRemote` support for Windows.

Could I ask you to add a patch to document this config setting?

>
> diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
> index efc732c0f31..9e2ea3b90cc 100644
> --- a/compat/fsmonitor/fsm-settings-darwin.c
> +++ b/compat/fsmonitor/fsm-settings-darwin.c
> @@ -2,10 +2,28 @@
>  #include "config.h"
>  #include "repository.h"
>  #include "fsmonitor-settings.h"
> +#include "fsmonitor-ipc.h"
>  #include "fsmonitor.h"
>  #include <sys/param.h>
>  #include <sys/mount.h>
>
> +/*
> + * Check if monitoring remote working directories is allowed.
> + *
> + * By default, monitoring remote working directories is
> + * disabled.  Users may override this behavior in enviroments where
> + * they have proper support.
> + */
> +static int check_config_allowremote(struct repository *r)
> +{
> +	int allow;
> +
> +	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
> +		return allow;
> +
> +	return -1; /* fsmonitor.allowremote not set */
> +}
> +
>  /*
>   * [1] Remote working directories are problematic for FSMonitor.
>   *
> @@ -27,24 +45,22 @@
>   * In theory, the above issues need to be addressed whether we are
>   * using the Hook or IPC API.
>   *
> + * So (for now at least), mark remote working directories as
> + * incompatible by default.
> + *

This was moved up, okay.

>   * For the builtin FSMonitor, we create the Unix domain socket for the
> - * IPC in the .git directory.  If the working directory is remote,
> - * then the socket will be created on the remote file system.  This
> - * can fail if the remote file system does not support UDS file types
> - * (e.g. smbfs to a Windows server) or if the remote kernel does not
> - * allow a non-local process to bind() the socket.  (These problems
> - * could be fixed by moving the UDS out of the .git directory and to a
> - * well-known local directory on the client machine, but care should
> - * be taken to ensure that $HOME is actually local and not a managed
> - * file share.)
> + * IPC in the temporary directory.  If the temporary directory is

This is incorrect. It is still the `.git` directory in the common case,
not a temporary directory.

> + * remote, then the socket will be created on the remote file system.
> + * This can fail if the remote file system does not support UDS file
> + * types (e.g. smbfs to a Windows server) or if the remote kernel does
> + * not allow a non-local process to bind() the socket.
>   *
> - * So (for now at least), mark remote working directories as
> - * incompatible.
> + * Therefore remote UDS locations are marked as incompatible.
>   *
>   *
>   * [2] FAT32 and NTFS working directories are problematic too.

Doesn't this patch address this, too? See below for more on that.

>   *
> - * The builtin FSMonitor uses a Unix domain socket in the .git
> + * The builtin FSMonitor uses a Unix domain socket in the temporary
>   * directory for IPC.  These Windows drive formats do not support
>   * Unix domain sockets, so mark them as incompatible for the daemon.
>   *
> @@ -65,6 +81,39 @@ static enum fsmonitor_reason check_volume(struct repository *r)
>  			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
>  			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
>
> +	if (!(fs.f_flags & MNT_LOCAL)) {
> +		switch (check_config_allowremote(r)) {
> +		case 0: /* config overrides and disables */
> +			return FSMONITOR_REASON_REMOTE;
> +		case 1: /* config overrides and enables */
> +			return FSMONITOR_REASON_OK;
> +		default:
> +			break; /* config has no opinion */
> +		}
> +
> +		return FSMONITOR_REASON_REMOTE;
> +	}

This `switch()` statement sounds like a verbose way to say the same as:

		return check_config_allowremote(r) == 1 ?
			FSMONITOR_REASON_OK : FSMONITOR_REASON_REMOTE;

> +
> +	return FSMONITOR_REASON_OK;
> +}
> +
> +static enum fsmonitor_reason check_uds_volume(void)

What's an UDS volume? Do you mean to say "Unix Domain Socket Volume"?

If so, it would be better to turn this into a function called
`filesystem_supports_unix_sockets()` and to return an `int`, 1 for "yes",
0 for "no".

> +{
> +	struct statfs fs;
> +	const char *path = fsmonitor_ipc__get_path();
> +
> +	if (statfs(path, &fs) == -1) {
> +		int saved_errno = errno;
> +		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
> +				 path, strerror(saved_errno));
> +		errno = saved_errno;
> +		return FSMONITOR_REASON_ERROR;
> +	}
> +
> +	trace_printf_key(&trace_fsmonitor,
> +			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
> +			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
> +
>  	if (!(fs.f_flags & MNT_LOCAL))
>  		return FSMONITOR_REASON_REMOTE;
>
> @@ -85,5 +134,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)

It is unfortunate that the diff hunk stops here, and mails lack a button
to increase the diff context. In this instance, the hidden part of the
`check_volume()` function is quite interesting: it returns
`FSMONITOR_REASON_NOSOCKETS` for `msdos` and `ntfs` file systems.

Which means that your patch changes behavior not only for remote file
systems, but also for local ones without support for Unix sockets.

To heed the principle of separation of concerns, please do split out that
part. I would recommend to make it the first patch to support
`msdos`/`ntfs` file systems (by registering the Unix sockets in a
temporary directory instead of the `.git/` directory). The second patch
can then introduce support for `fsmonitor.allowRemote` on macOS on top of
the first patch.

>  	if (reason != FSMONITOR_REASON_OK)
>  		return reason;
>
> +	reason = check_uds_volume();
> +	if (reason != FSMONITOR_REASON_OK)
> +		return reason;
> +
>  	return FSMONITOR_REASON_OK;
>  }
> diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
> index 789e7397baa..6e9b40a03d5 100644
> --- a/fsmonitor-ipc.c
> +++ b/fsmonitor-ipc.c
> @@ -4,6 +4,7 @@
>  #include "fsmonitor-ipc.h"
>  #include "run-command.h"
>  #include "strbuf.h"
> +#include "tempfile.h"
>  #include "trace2.h"
>
>  #ifndef HAVE_FSMONITOR_DAEMON_BACKEND
> @@ -47,7 +48,51 @@ int fsmonitor_ipc__is_supported(void)
>  	return 1;
>  }
>
> -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
> +GIT_PATH_FUNC(fsmonitor_ipc__get_pathfile, "fsmonitor--daemon.ipc")

Why rename this? That's unnecessary chatter in the patch. Let's avoid such
things in the future, it only costs reviewers time.

> +
> +static char *gen_ipc_file(void)
> +{
> +	char *retval = NULL;
> +	struct tempfile *ipc;
> +
> +	const char *ipc_file = fsmonitor_ipc__get_pathfile();
> +	FILE *fp = fopen(ipc_file, "w");
> +
> +	if (!fp)
> +		die_errno("error opening '%s'", ipc_file);
> +	ipc = mks_tempfile_t("fsmonitor_ipc_XXXXXX");
> +	strbuf_write(&ipc->filename, fp);
> +	fclose(fp);
> +	retval = strbuf_detach(&ipc->filename, NULL);
> +	strbuf_release(&ipc->filename);
> +	return retval;
> +}
> +
> +const char *fsmonitor_ipc__get_path(void)
> +{
> +	char *retval = NULL;
> +	struct strbuf sb = STRBUF_INIT;
> +
> +	const char *ipc_file = fsmonitor_ipc__get_pathfile();
> +	FILE *fp = fopen(ipc_file, "r");
> +
> +	if (!fp) {
> +		return gen_ipc_file();
> +	} else {
> +		strbuf_read(&sb, fileno(fp), 0);
> +		fclose(fp);
> +		fp = fopen(sb.buf, "r");
> +		if (!fp) { /* generate new file */
> +			if (unlink(ipc_file) < 0)
> +				die_errno("could not remove '%s'", ipc_file);
> +			return gen_ipc_file();
> +		}
> +		fclose(fp);
> +		retval = strbuf_detach(&sb, NULL);
> +		strbuf_release(&sb);
> +		return retval;
> +	}
> +}

I am afraid I do not understand how this code can guarantee a fixed path
for the Unix domain socket.

It _needs_ to be fixed so that a singleton daemon can run and listen on
it, and an arbitrary number of Git clients can connect to it.

If it is not fixed, you will cause Git to quite possibly start a new
FSMonitor daemon for every invocation that wants to connect to an
FSMonitor daemon.

This means that the path of the Unix socket needs to have a 1:1
relationship to the path of the `.git/` directory. If you install it in
that directory, that invariant is naturally fulfilled. If you want to
install it elsewhere, you will have to come up with a reliable way to
guarantee that connection.

One option would be to install the Unix sockets in the home directory,
under a name like `.git-fsmonitor-<hash>` where the <hash> is e.g. a
SHA-1/SHA-256 of the canonicalized path of the `.git/` directory.

>
>  enum ipc_active_state fsmonitor_ipc__get_state(void)
>  {
> diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
> index b6a7067c3af..63277dea39e 100644
> --- a/fsmonitor-ipc.h
> +++ b/fsmonitor-ipc.h
> @@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
>   */
>  const char *fsmonitor_ipc__get_path(void);
>
> +/*
> + * Returns the pathname to the file that contains the pathname to the
> + * IPC named pipe or Unix domain socket.
> + */
> +const char *fsmonitor_ipc__get_pathfile(void);
> +
>  /*
>   * Try to determine whether there is a `git-fsmonitor--daemon` process
>   * listening on the IPC pipe/socket.

Thank you for working on this, also on the Windows side. It definitely
helps!

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-08-18 21:35 ` Junio C Hamano
  2022-08-19 10:05 ` Johannes Schindelin
@ 2022-08-19 16:50 ` Jeff Hostetler
  2022-08-19 18:38   ` Eric DeCosta
  2022-08-19 17:48 ` Eric Sunshine
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-08-19 16:50 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git; +Cc: Eric DeCosta



On 8/18/22 4:48 PM, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>
> 
> Though perhaps not common, there are uses cases where users have large,
> network-mounted repos. Having the ability to run fsmonitor against
> network paths would benefit those users.
> 
> As a first step towards enabling fsmonitor to work against
> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> was introduced for Windows. Setting this option to true will override
> the default behavior (erroring-out) when a network-mounted repo is
> detected by fsmonitor. In order for macOS to have parity with Windows,
> the same option is now introduced for macOS.

We might also say that this config option only allows FSMonitor
to TRY to consider using a network-mounted repo.  And that this
ability is considered experimental until sufficient testing can
be completed and we can determine the combinations of
{ client os } x { server os } x { remote access } x { file system type }
that are known to work or not work and we can update the defaults
and the documentation accordingly.

For example, on a MacOS client, we expect the local "fseventsd" service
to send us recursive events on all files and sub directories under the
repo root.  If the server is a Linux machine (which doesn't really do
recursive events), does exporting the FS from the server over NFS or SMB
(or whatever) cause the Linux host to send enough information to the
client machine for fseventsd to synthesize the recursive event stream
locally that FSMonitor expects.  It might.  It might not.  That
combination should be tested (along with a lot of other combinations).

But again, this patch is just about allowing the (informed?) user to
try it and begin testing various combinations.


> 
> The the added wrinkle being that the Unix domain socket (UDS) file
> used for IPC cannot be created in a network location; instead the
> temporary directory is used.

This scares me a bit.  I put the socket in the .git directory
so that we are guaranteed that only one daemon will run on the
repository and that all clients will know where to find that socket
(if it exists).

It looks like you're creating the UDS using a tmp pathname and
writing the pathname to the actual .git/fsmonitor--daemon.ipc FILE.
This adds a layer of indirection and is prone to races.


The act of creating the actual socket is protected by code in
unix-socket.c and unix-stream-server.c to try to safely create
the socket and avoid stepping on another active daemon (who
currently has the server-side of the socket open).

My code also detects dead sockets (where a previous daemon died
and failed to delete the socket).


Additionally, allowing remote access means that the repo could
be used by multiple client machines and/or by the server machine
itself.  Consider the example of two MacOS clients mounting the
remote repo and both wanting to start FSMonitor.  They would
constantly fight to recreate a new local-tmp-based socket and
update your pathname FILE and end up invalidating each other on
each command.


Also, if someone overwrites your new pathname FILE, but doesn't tell
the daemon, the daemon will be orphaned -- still running, but no one
will ever connect to it because the FILE no longer points to it.


There was a suggestion later in this thread about using a SHA-1
or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern
and just put the socket in $HOME (and omit the need for the new
fsmonitor-daemon.ipc FILE completely).  This might work, but we
need to be careful because a user might have hardlinks or symlinks
locally so there may be more than one unique path to the repo
on the local system.  (It is OK to have more than one daemon
listening to a repo, just less efficient.)


As an interim step, you might try using my original socket code
plus just the config.allowRemote=true change.  And test it on a
mounted repo where you've converted the .git directory to a .git
file and moved contents of the .git directory to somewhere local.
Then the UDS would be created in the local GITDIR instead of on
the remote system.  This won't help any of the sharing cases I
described above, but will let you experiment with getting remote
events.

Jeff



> base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
> 

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                   ` (2 preceding siblings ...)
  2022-08-19 16:50 ` Jeff Hostetler
@ 2022-08-19 17:48 ` Eric Sunshine
  2022-08-19 18:58 ` Torsten Bögershausen
                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric Sunshine @ 2022-08-19 17:48 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: Git List, Eric DeCosta

On Thu, Aug 18, 2022 at 5:00 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Though perhaps not common, there are uses cases where users have large,

s/uses cases/use cases/

> network-mounted repos. Having the ability to run fsmonitor against
> network paths would benefit those users.
>
> As a first step towards enabling fsmonitor to work against
> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> was introduced for Windows. Setting this option to true will override
> the default behavior (erroring-out) when a network-mounted repo is
> detected by fsmonitor. In order for macOS to have parity with Windows,
> the same option is now introduced for macOS.
>
> The the added wrinkle being that the Unix domain socket (UDS) file

s/The the/The/

> used for IPC cannot be created in a network location; instead the
> temporary directory is used.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-19 16:50 ` Jeff Hostetler
@ 2022-08-19 18:38   ` Eric DeCosta
  2022-08-19 20:15     ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta @ 2022-08-19 18:38 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Friday, August 19, 2022 12:50 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org
> Cc: Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH] fsmonitor: option to allow fsmonitor to run against
> network-mounted repos
> 
> 
> 
> On 8/18/22 4:48 PM, Eric DeCosta via GitGitGadget wrote:
> > From: Eric DeCosta <edecosta@mathworks.com>
> >
> > Though perhaps not common, there are uses cases where users have large,
> > network-mounted repos. Having the ability to run fsmonitor against
> > network paths would benefit those users.
> >
> > As a first step towards enabling fsmonitor to work against
> > network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> > was introduced for Windows. Setting this option to true will override
> > the default behavior (erroring-out) when a network-mounted repo is
> > detected by fsmonitor. In order for macOS to have parity with Windows,
> > the same option is now introduced for macOS.
> 
> We might also say that this config option only allows FSMonitor
> to TRY to consider using a network-mounted repo.  And that this
> ability is considered experimental until sufficient testing can
> be completed and we can determine the combinations of
> { client os } x { server os } x { remote access } x { file system type }
> that are known to work or not work and we can update the defaults
> and the documentation accordingly.
> 
Yes, very experimental. 

> For example, on a MacOS client, we expect the local "fseventsd" service
> to send us recursive events on all files and sub directories under the
> repo root.  If the server is a Linux machine (which doesn't really do
> recursive events), does exporting the FS from the server over NFS or SMB
> (or whatever) cause the Linux host to send enough information to the
> client machine for fseventsd to synthesize the recursive event stream
> locally that FSMonitor expects.  It might.  It might not.  That
> combination should be tested (along with a lot of other combinations).
> 
> But again, this patch is just about allowing the (informed?) user to
> try it and begin testing various combinations.
> 
Yes, the point is to allow users to try it out.  Self-servingly, I have about
3K users who make heavy use of network-mounted sandboxes on the
three major platforms; all connecting via NFS or SMB to Linux file
servers.  Hardly exhaustive, but the file system change notification APIs
(inotify, FSEvents, and ReadDirectoryCHangesW) all seem to work correctly.
Thus my motivation to work on this aspect of git :-)

> 
> >
> > The the added wrinkle being that the Unix domain socket (UDS) file
> > used for IPC cannot be created in a network location; instead the
> > temporary directory is used.
> 
> This scares me a bit.  I put the socket in the .git directory
> so that we are guaranteed that only one daemon will run on the
> repository and that all clients will know where to find that socket
> (if it exists).
> 
> It looks like you're creating the UDS using a tmp pathname and
> writing the pathname to the actual .git/fsmonitor--daemon.ipc FILE.
> This adds a layer of indirection and is prone to races.
> 
Good point.

> 
> The act of creating the actual socket is protected by code in
> unix-socket.c and unix-stream-server.c to try to safely create
> the socket and avoid stepping on another active daemon (who
> currently has the server-side of the socket open).
> 
> My code also detects dead sockets (where a previous daemon died
> and failed to delete the socket).
> 
> 
> Additionally, allowing remote access means that the repo could
> be used by multiple client machines and/or by the server machine
> itself.  Consider the example of two MacOS clients mounting the
> remote repo and both wanting to start FSMonitor.  They would
> constantly fight to recreate a new local-tmp-based socket and
> update your pathname FILE and end up invalidating each other on
> each command.
> 
I see your point - they'd stomp on each other. 

As far as multiple client machines mounting the remote repo, I have
doubts that FSMonitor would even see changes made from another
machine. Worth trying out and documenting as needed - might even
be better off being considered as unsupported.

> 
> Also, if someone overwrites your new pathname FILE, but doesn't tell
> the daemon, the daemon will be orphaned -- still running, but no one
> will ever connect to it because the FILE no longer points to it.
> 
True, thanks for pointing that out.

> 
> There was a suggestion later in this thread about using a SHA-1
> or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern
> and just put the socket in $HOME (and omit the need for the new
> fsmonitor-daemon.ipc FILE completely).  This might work, but we
> need to be careful because a user might have hardlinks or symlinks
> locally so there may be more than one unique path to the repo
> on the local system.  (It is OK to have more than one daemon
> listening to a repo, just less efficient.)
> 
Ah, I see.

> 
> As an interim step, you might try using my original socket code
> plus just the config.allowRemote=true change.  And test it on a
> mounted repo where you've converted the .git directory to a .git
> file and moved contents of the .git directory to somewhere local.
> Then the UDS would be created in the local GITDIR instead of on
> the remote system.  This won't help any of the sharing cases I
> described above, but will let you experiment with getting remote
> events.
> 
Within the context of the environment that I have available to me
(macOS over NFS to a Linux file server), FSEvents is working correctly.
I can make changes at any arbitrary place inside of the repo and an
event is generated. 

It's looking like that the Unix domain socket (UDS) file should remain
where it is unless fsmonitor.allowRemote is true.

If fsmonitor.allowRemote is true then the UDS file can be located
in $HOME with the caveat that if there is more than one path to
the repo (via hard or sym links) that things might not work as
expected.  I think that's OK given the experimental nature of the
feature.

-Eric


> Jeff
> 
> 
> 
> > base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
> >


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                   ` (3 preceding siblings ...)
  2022-08-19 17:48 ` Eric Sunshine
@ 2022-08-19 18:58 ` Torsten Bögershausen
  2022-08-20 22:24 ` Junio C Hamano
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
  6 siblings, 0 replies; 157+ messages in thread
From: Torsten Bögershausen @ 2022-08-19 18:58 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

On Thu, Aug 18, 2022 at 08:48:23PM +0000, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>

Just some comments on the commit message, please see inline.

>
> Though perhaps not common, there are uses cases where users have large,
> network-mounted repos. Having the ability to run fsmonitor against
> network paths would benefit those users.

I think that we can drop the "Though perhaps not common" - it doesn't
add any information about the technical part of the patch.
(And that is, what is important)

And I personally use network-mounted repos every day ;-)

Sone users have large, network-mounted repos. Having the ability
to run fsmonitor against network paths would benefit those users.

>
> As a first step towards enabling fsmonitor to work against
> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> was introduced for Windows. Setting this option to true will override
> the default behavior (erroring-out) when a network-mounted repo is
> detected by fsmonitor.
[]
The same option is now introduced for macOS.

>
> The the added wrinkle being that the Unix domain socket (UDS) file
  ^^^ ^^^

> used for IPC cannot be created in a network location; instead the
> temporary directory is used.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-19 18:38   ` Eric DeCosta
@ 2022-08-19 20:15     ` Jeff Hostetler
  0 siblings, 0 replies; 157+ messages in thread
From: Jeff Hostetler @ 2022-08-19 20:15 UTC (permalink / raw)
  To: Eric DeCosta, Eric DeCosta via GitGitGadget, git



On 8/19/22 2:38 PM, Eric DeCosta wrote:
> 
> 
>> -----Original Message-----
>> From: Jeff Hostetler <git@jeffhostetler.com>
>> Sent: Friday, August 19, 2022 12:50 PM
>> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
>> git@vger.kernel.org
>> Cc: Eric DeCosta <edecosta@mathworks.com>
>> Subject: Re: [PATCH] fsmonitor: option to allow fsmonitor to run against
>> network-mounted repos
>>
>>
>>
>> On 8/18/22 4:48 PM, Eric DeCosta via GitGitGadget wrote:
>>> From: Eric DeCosta <edecosta@mathworks.com>
>>>

[...]
> Yes, the point is to allow users to try it out.  Self-servingly, I have about
> 3K users who make heavy use of network-mounted sandboxes on the
> three major platforms; all connecting via NFS or SMB to Linux file
> servers.  Hardly exhaustive, but the file system change notification APIs
> (inotify, FSEvents, and ReadDirectoryCHangesW) all seem to work correctly.
> Thus my motivation to work on this aspect of git :-)

Perfect.  I have to be careful here, I don't want to discourage
experimentation and testing -- and if you have access to a pool of
users who can beat on it, then great let's try it.

At the time, I didn't have such a "captive audience"... :-)

And I just wanted to be sure that everyone understood that some of
these combinations have never been tried....

[...]
> As far as multiple client machines mounting the remote repo, I have
> doubts that FSMonitor would even see changes made from another
> machine. Worth trying out and documenting as needed - might even
> be better off being considered as unsupported.

Yeah, that's another dimension.  (1) is whether we miss events
that we generated locally.  (2) is whether events get relayed to
us from other clients.


>> There was a suggestion later in this thread about using a SHA-1
>> or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern
>> and just put the socket in $HOME (and omit the need for the new
>> fsmonitor-daemon.ipc FILE completely).  This might work, but we
>> need to be careful because a user might have hardlinks or symlinks
>> locally so there may be more than one unique path to the repo
>> on the local system.  (It is OK to have more than one daemon
>> listening to a repo, just less efficient.)
>>
> Ah, I see.
> 
>>
>> As an interim step, you might try using my original socket code
>> plus just the config.allowRemote=true change.  And test it on a
>> mounted repo where you've converted the .git directory to a .git
>> file and moved contents of the .git directory to somewhere local.
>> Then the UDS would be created in the local GITDIR instead of on
>> the remote system.  This won't help any of the sharing cases I
>> described above, but will let you experiment with getting remote
>> events.
>>
> Within the context of the environment that I have available to me
> (macOS over NFS to a Linux file server), FSEvents is working correctly.
> I can make changes at any arbitrary place inside of the repo and an
> event is generated.
> 
> It's looking like that the Unix domain socket (UDS) file should remain
> where it is unless fsmonitor.allowRemote is true.
> 
> If fsmonitor.allowRemote is true then the UDS file can be located
> in $HOME with the caveat that if there is more than one path to
> the repo (via hard or sym links) that things might not work as
> expected.  I think that's OK given the experimental nature of the
> feature.

Yeah, I'd experiment with moving the UDS to $HOME (assuming you don't
remote mount home directories too) and see how it works for you.
Later, after we have more experience with it, we can talk about just
having it always be in $HOME.

Jeff


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                   ` (4 preceding siblings ...)
  2022-08-19 18:58 ` Torsten Bögershausen
@ 2022-08-20 22:24 ` Junio C Hamano
  2022-08-22 13:22   ` Johannes Schindelin
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
  6 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-08-20 22:24 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

> As a first step towards enabling fsmonitor to work against
> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> was introduced for Windows. Setting this option to true will override
> the default behavior (erroring-out) when a network-mounted repo is
> detected by fsmonitor. In order for macOS to have parity with Windows,
> the same option is now introduced for macOS.

With this merged in, recent CI runs for 'seen'

e.g. https://github.com/git/git/actions/runs/2892889122

seems to break macOS jobs, letting them hog CPU forever and exceed
6hr or whatever the limit is.

As an experiment I pushed out 'seen' but without this commit (not
the entire topic is excluded, the Windows one is still included).
As there is nothing specific to macOS between 'next' and 'seen',
macOS jobs seem to pass, which is not very surprising.

https://github.com/git/git/actions/runs/2896207171

As the patch collected some review comments, I've already marked it
in the "What's cooking" draft as expecting a reroll of that step;
until that happens, let's keep it out of 'seen'.

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-20 22:24 ` Junio C Hamano
@ 2022-08-22 13:22   ` Johannes Schindelin
  2022-08-22 16:07     ` Junio C Hamano
  2022-08-23 13:51     ` Jeff Hostetler
  0 siblings, 2 replies; 157+ messages in thread
From: Johannes Schindelin @ 2022-08-22 13:22 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Eric DeCosta via GitGitGadget, git, Eric DeCosta

Hi Junio,

On Sat, 20 Aug 2022, Junio C Hamano wrote:

> > As a first step towards enabling fsmonitor to work against
> > network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
> > was introduced for Windows. Setting this option to true will override
> > the default behavior (erroring-out) when a network-mounted repo is
> > detected by fsmonitor. In order for macOS to have parity with Windows,
> > the same option is now introduced for macOS.
>
> With this merged in, recent CI runs for 'seen'
>
> e.g. https://github.com/git/git/actions/runs/2892889122
>
> seems to break macOS jobs, letting them hog CPU forever and exceed
> 6hr or whatever the limit is.
>
> As an experiment I pushed out 'seen' but without this commit (not
> the entire topic is excluded, the Windows one is still included).
> As there is nothing specific to macOS between 'next' and 'seen',
> macOS jobs seem to pass, which is not very surprising.
>
> https://github.com/git/git/actions/runs/2896207171
>
> As the patch collected some review comments, I've already marked it
> in the "What's cooking" draft as expecting a reroll of that step;
> until that happens, let's keep it out of 'seen'.

It makes sense to keep it out of `seen`, and at the same time I would like
to encourage Eric to investigate what causes those time-outs.

When toggling timestamps (click on the wheel on the upper right) at
https://github.com/git/git/runs/7927812510?check_suite_focus=true#step:4:1774,
it can be seen that at close to 1am, t9903 finished, but then nothing
happens until twenty past 6am.

I've downloaded the raw logs (also available via the wheel on the upper
right) to find out which test timed out:

	$ diff -u \
	  <(sed -n 's/.*\] \(t[0-9][^ ]*\).*/\1/p' <~/Downloads/17 | sort) \
	  <(git ls-tree upstream/seen:t | cut -c 54- | grep '^t[0-9].*-.*sh$')

	--- /dev/fd/63  2022-08-22 14:56:05.510269527 +0200
	+++ /dev/fd/62  2022-08-22 14:56:05.510269527 +0200
	@@ -794,6 +794,7 @@
	 t7524-commit-summary.sh
	 t7525-status-rename.sh
	 t7526-commit-pathspec-file.sh
	+t7527-builtin-fsmonitor.sh
	 t7528-signed-commit-ssh.sh
	 t7600-merge.sh
	 t7601-merge-pull-config.sh
	@@ -945,6 +946,7 @@
	 t9812-git-p4-wildcards.sh
	 t9813-git-p4-preserve-users.sh
	 t9814-git-p4-rename.sh
	+t9815-git-p4-submit-fail.sh
	 t9816-git-p4-locked.sh
	 t9817-git-p4-exclude.sh
	 t9818-git-p4-block.sh
	@@ -964,5 +966,8 @@
	 t9832-unshelve.sh
	 t9833-errors.sh
	 t9834-git-p4-file-dir-bug.sh
	+t9835-git-p4-metadata-encoding-python2.sh
	+t9836-git-p4-metadata-encoding-python3.sh
	 t9901-git-web--browse.sh
	+t9902-completion.sh
	 t9903-bash-prompt.sh

I have no idea what's up with t98* and t9902, but I would bet that they
were somehow "swallowed" by `prove` being terminated, and that the
actual test script that times out is t7527.

Eric: To investigate, you will want to reproduce the problem on a macOS
machine. If you have none available, you could create a temporary branch,
heavily edit the CI definition, and push it to GitHub. And by heavy edits
I mean something like this:

- Remove all non-macOS jobs from `.github/workflows/main.yml` (that means
  removing all but the `regular` job, removing all but at least one
  `macos` matrix entry, and removing the the `needs: ci-config` and
  corresponding `if:` line.

- Edit `t/Makefile` to define `T = t7527-builtin-fsmonitor.sh` instead of
  running all the tests.

- Edit `.github/workflows/main.yml` so that the step that causes the
  time-out has a chance of timing out much sooner (and the subsequent
  steps then have a chance to upload the relevant logs):
  https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes

If this does not shed any light into the issue, please let me know, I have
a couple more aces up my sleeve.

Ciao,
Dscho

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-22 13:22   ` Johannes Schindelin
@ 2022-08-22 16:07     ` Junio C Hamano
  2022-08-23 13:51     ` Jeff Hostetler
  1 sibling, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-08-22 16:07 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Eric DeCosta via GitGitGadget, git, Eric DeCosta

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Eric: To investigate, you will want to reproduce the problem on a macOS
> machine. If you have none available, you could create a temporary branch,
> heavily edit the CI definition, and push it to GitHub. And by heavy edits
> I mean something like this:
>
> - Remove all non-macOS jobs from `.github/workflows/main.yml` (that means
>   removing all but the `regular` job, removing all but at least one
>   `macos` matrix entry, and removing the the `needs: ci-config` and
>   corresponding `if:` line.
>
> - Edit `t/Makefile` to define `T = t7527-builtin-fsmonitor.sh` instead of
>   running all the tests.
>
> - Edit `.github/workflows/main.yml` so that the step that causes the
>   time-out has a chance of timing out much sooner (and the subsequent
>   steps then have a chance to upload the relevant logs):
>   https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes
>
> If this does not shed any light into the issue, please let me know, I have
> a couple more aces up my sleeve.

Thanks, both.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v2 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-18 20:48 [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                   ` (5 preceding siblings ...)
  2022-08-20 22:24 ` Junio C Hamano
@ 2022-08-23 13:03 ` Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 1/4] " Eric DeCosta via GitGitGadget
                     ` (4 more replies)
  6 siblings, 5 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-23 13:03 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta

cc: Johannes Schindelin Johannes.Schindelin@gmx.de cc: Jeff Hostetler
git@jeffhostetler.com cc: Eric Sunshine sunshine@sunshineco.com cc: Torsten
Bögershausen tboegi@web.de

Eric DeCosta (2):
  fsmonitor: option to allow fsmonitor to run against network-mounted
    repos
  fsmonitor: macOS: allow fsmonitor to run against network-mounted repos

edecosta (2):
  Check working directory and Unix domain socket file for compatability
  Minor refactoring and simplification of Windows settings checks

 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 compat/fsmonitor/fsm-settings-win32.c  | 47 ++++++++++++++++-
 fsmonitor-ipc.c                        | 40 ++++++++++++--
 fsmonitor-ipc.h                        |  6 +++
 fsmonitor-settings.c                   | 67 +++++++++++++++++++++++-
 fsmonitor-settings.h                   |  4 ++
 6 files changed, 210 insertions(+), 26 deletions(-)


base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v1:

 -:  ----------- > 1:  40ce21e85c9 fsmonitor: option to allow fsmonitor to run against network-mounted repos
 -:  ----------- > 2:  46b4efd96cc fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
 1:  425df16ac0b ! 3:  9b128a98149 fsmonitor: option to allow fsmonitor to run against network-mounted repos
     @@
       ## Metadata ##
     -Author: Eric DeCosta <edecosta@mathworks.com>
     +Author: edecosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    fsmonitor: option to allow fsmonitor to run against network-mounted repos
     +    Check working directory and Unix domain socket file for compatability
      
     -    Though perhaps not common, there are uses cases where users have large,
     -    network-mounted repos. Having the ability to run fsmonitor against
     -    network paths would benefit those users.
     +    Perform separate checks for the working directory and Unix domain socket
     +    (UDS) file location. The working directory may be located on a
     +    network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS
     +    file may never be located on a network-mounted file system; additionally
     +    it may not be located on FAT32 or NTFS file systems.
      
     -    As a first step towards enabling fsmonitor to work against
     -    network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
     -    was introduced for Windows. Setting this option to true will override
     -    the default behavior (erroring-out) when a network-mounted repo is
     -    detected by fsmonitor. In order for macOS to have parity with Windows,
     -    the same option is now introduced for macOS.
     -
     -    The the added wrinkle being that the Unix domain socket (UDS) file
     -    used for IPC cannot be created in a network location; instead the
     -    temporary directory is used.
     -
     -    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
     +    Signed-off-by: edecosta <edecosta@mathworks.com>
      
       ## compat/fsmonitor/fsm-settings-darwin.c ##
      @@
     - #include "config.h"
       #include "repository.h"
       #include "fsmonitor-settings.h"
     -+#include "fsmonitor-ipc.h"
       #include "fsmonitor.h"
     ++#include "fsmonitor-ipc.h"
       #include <sys/param.h>
       #include <sys/mount.h>
       
     -+/*
     -+ * Check if monitoring remote working directories is allowed.
     -+ *
     -+ * By default, monitoring remote working directories is
     -+ * disabled.  Users may override this behavior in enviroments where
     -+ * they have proper support.
     -+ */
     -+static int check_config_allowremote(struct repository *r)
     -+{
     -+	int allow;
     -+
     -+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
     -+		return allow;
     -+
     -+	return -1; /* fsmonitor.allowremote not set */
     -+}
     -+
       /*
     -  * [1] Remote working directories are problematic for FSMonitor.
     +- * [1] Remote working directories are problematic for FSMonitor.
     ++ * Remote working directories are problematic for FSMonitor.
        *
     +  * The underlying file system on the server machine and/or the remote
     +  * mount type (NFS, SAMBA, etc.) dictates whether notification events
      @@
        * In theory, the above issues need to be addressed whether we are
        * using the Hook or IPC API.
        *
     -+ * So (for now at least), mark remote working directories as
     -+ * incompatible by default.
     -+ *
     -  * For the builtin FSMonitor, we create the Unix domain socket for the
     +- * For the builtin FSMonitor, we create the Unix domain socket for the
      - * IPC in the .git directory.  If the working directory is remote,
      - * then the socket will be created on the remote file system.  This
      - * can fail if the remote file system does not support UDS file types
     @@ compat/fsmonitor/fsm-settings-darwin.c
      - * well-known local directory on the client machine, but care should
      - * be taken to ensure that $HOME is actually local and not a managed
      - * file share.)
     -+ * IPC in the temporary directory.  If the temporary directory is
     -+ * remote, then the socket will be created on the remote file system.
     -+ * This can fail if the remote file system does not support UDS file
     -+ * types (e.g. smbfs to a Windows server) or if the remote kernel does
     -+ * not allow a non-local process to bind() the socket.
     -  *
     -- * So (for now at least), mark remote working directories as
     +- *
     +  * So (for now at least), mark remote working directories as
      - * incompatible.
     -+ * Therefore remote UDS locations are marked as incompatible.
     -  *
     -  *
     -  * [2] FAT32 and NTFS working directories are problematic too.
     -  *
     +- *
     +- *
     +- * [2] FAT32 and NTFS working directories are problematic too.
     +- *
      - * The builtin FSMonitor uses a Unix domain socket in the .git
     -+ * The builtin FSMonitor uses a Unix domain socket in the temporary
     -  * directory for IPC.  These Windows drive formats do not support
     -  * Unix domain sockets, so mark them as incompatible for the daemon.
     +- * directory for IPC.  These Windows drive formats do not support
     +- * Unix domain sockets, so mark them as incompatible for the daemon.
     ++ * incompatible unless fsmonitor.allowRemote is true.
        *
     +  */
     + static enum fsmonitor_reason check_volume(struct repository *r)
      @@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_volume(struct repository *r)
       			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
       			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
       
     -+	if (!(fs.f_flags & MNT_LOCAL)) {
     -+		switch (check_config_allowremote(r)) {
     -+		case 0: /* config overrides and disables */
     ++	if (!(fs.f_flags & MNT_LOCAL)
     ++		&& (fsm_settings__get_allow_remote(r) < 1))
      +			return FSMONITOR_REASON_REMOTE;
     -+		case 1: /* config overrides and enables */
     -+			return FSMONITOR_REASON_OK;
     -+		default:
     -+			break; /* config has no opinion */
     -+		}
     -+
     -+		return FSMONITOR_REASON_REMOTE;
     -+	}
      +
      +	return FSMONITOR_REASON_OK;
      +}
      +
     ++/*
     ++ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
     ++ * for the IPC in the .git directory by default or $HOME if
     ++ * fsmonitor.allowRemote is true.  If the directory is remote,
     ++ * then the socket will be created on the remote file system. This
     ++ * can fail if the remote file system does not support UDS file types
     ++ * (e.g. smbfs to a Windows server) or if the remote kernel does not
     ++ * allow a non-local process to bind() the socket.
     ++ *
     ++ * Therefore remote UDS locations are marked as incompatible.
     ++ *
     ++ * FAT32 and NTFS working directories are problematic too.
     ++ *
     ++ * These Windows drive formats do not support Unix domain sockets, so
     ++ * mark them as incompatible for the location of the UDS file.
     ++ *
     ++ */
      +static enum fsmonitor_reason check_uds_volume(void)
      +{
      +	struct statfs fs;
     -+	const char *path = fsmonitor_ipc__get_path();
     ++	struct strbuf path = STRBUF_INIT;
     ++	const char *ipc_path = fsmonitor_ipc__get_path();
     ++	strbuf_add(&path, ipc_path, strlen(ipc_path));
      +
     -+	if (statfs(path, &fs) == -1) {
     ++	if (statfs(dirname(path.buf), &fs) == -1) {
      +		int saved_errno = errno;
      +		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
     -+				 path, strerror(saved_errno));
     ++				 path.buf, strerror(saved_errno));
      +		errno = saved_errno;
     ++		strbuf_release(&path);
      +		return FSMONITOR_REASON_ERROR;
      +	}
      +
      +	trace_printf_key(&trace_fsmonitor,
      +			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
     -+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
     ++			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
     ++	strbuf_release(&path);
      +
       	if (!(fs.f_flags & MNT_LOCAL))
       		return FSMONITOR_REASON_REMOTE;
     @@ compat/fsmonitor/fsm-settings-darwin.c: enum fsmonitor_reason fsm_os__incompatib
      +
       	return FSMONITOR_REASON_OK;
       }
     -
     - ## fsmonitor-ipc.c ##
     -@@
     - #include "fsmonitor-ipc.h"
     - #include "run-command.h"
     - #include "strbuf.h"
     -+#include "tempfile.h"
     - #include "trace2.h"
     - 
     - #ifndef HAVE_FSMONITOR_DAEMON_BACKEND
     -@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
     - 	return 1;
     - }
     - 
     --GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     -+GIT_PATH_FUNC(fsmonitor_ipc__get_pathfile, "fsmonitor--daemon.ipc")
     -+
     -+static char *gen_ipc_file(void)
     -+{
     -+	char *retval = NULL;
     -+	struct tempfile *ipc;
     -+
     -+	const char *ipc_file = fsmonitor_ipc__get_pathfile();
     -+	FILE *fp = fopen(ipc_file, "w");
     -+
     -+	if (!fp)
     -+		die_errno("error opening '%s'", ipc_file);
     -+	ipc = mks_tempfile_t("fsmonitor_ipc_XXXXXX");
     -+	strbuf_write(&ipc->filename, fp);
     -+	fclose(fp);
     -+	retval = strbuf_detach(&ipc->filename, NULL);
     -+	strbuf_release(&ipc->filename);
     -+	return retval;
     -+}
     -+
     -+const char *fsmonitor_ipc__get_path(void)
     -+{
     -+	char *retval = NULL;
     -+	struct strbuf sb = STRBUF_INIT;
     -+
     -+	const char *ipc_file = fsmonitor_ipc__get_pathfile();
     -+	FILE *fp = fopen(ipc_file, "r");
     -+
     -+	if (!fp) {
     -+		return gen_ipc_file();
     -+	} else {
     -+		strbuf_read(&sb, fileno(fp), 0);
     -+		fclose(fp);
     -+		fp = fopen(sb.buf, "r");
     -+		if (!fp) { /* generate new file */
     -+			if (unlink(ipc_file) < 0)
     -+				die_errno("could not remove '%s'", ipc_file);
     -+			return gen_ipc_file();
     -+		}
     -+		fclose(fp);
     -+		retval = strbuf_detach(&sb, NULL);
     -+		strbuf_release(&sb);
     -+		return retval;
     -+	}
     -+}
     - 
     - enum ipc_active_state fsmonitor_ipc__get_state(void)
     - {
     -
     - ## fsmonitor-ipc.h ##
     -@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
     -  */
     - const char *fsmonitor_ipc__get_path(void);
     - 
     -+/*
     -+ * Returns the pathname to the file that contains the pathname to the
     -+ * IPC named pipe or Unix domain socket.
     -+ */
     -+const char *fsmonitor_ipc__get_pathfile(void);
     -+
     - /*
     -  * Try to determine whether there is a `git-fsmonitor--daemon` process
     -  * listening on the IPC pipe/socket.
 -:  ----------- > 4:  15c965801f8 Minor refactoring and simplification of Windows settings checks

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v2 1/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
@ 2022-08-23 13:03   ` Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 2/4] fsmonitor: macOS: " Eric DeCosta via GitGitGadget
                     ` (3 subsequent siblings)
  4 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-23 13:03 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Though perhaps not common, there are use cases where users have large,
network-mounted repos. Having the ability to run fsmonitor against
network paths would benefit those users.

Most modern Samba-based filers have the necessary support to enable
fsmonitor on network-mounted repos. As a first step towards enabling
fsmonitor to work against network-mounted repos, introduce a
configuration option, 'fsmonitor.allowRemote'. Setting this option to
true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-win32.c | 68 +++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index 907655720bb..e5ec5b0a9f7 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
+/*
+ * Check if monitoring remote working directories is allowed.
+ *
+ * By default, monitoring remote working directories is
+ * disabled.  Users may override this behavior in enviroments where
+ * they have proper support.
+ */
+static int check_config_allowremote(struct repository *r)
+{
+	int allow;
+
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		return allow;
+
+	return -1; /* fsmonitor.allowremote not set */
+}
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Error if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
 /*
  * Remote working directories are problematic for FSMonitor.
  *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
  */
 static enum fsmonitor_reason check_remote(struct repository *r)
 {
+	int ret;
 	wchar_t wpath[MAX_PATH];
 	wchar_t wfullpath[MAX_PATH];
 	size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 		trace_printf_key(&trace_fsmonitor,
 				 "check_remote('%s') true",
 				 r->worktree);
+
+		ret = check_remote_protocol(wfullpath);
+		if (ret < 0)
+			return FSMONITOR_REASON_ERROR;
+
+		switch (check_config_allowremote(r)) {
+		case 0: /* config overrides and disables */
+			return FSMONITOR_REASON_REMOTE;
+		case 1: /* config overrides and enables */
+			return FSMONITOR_REASON_OK;
+		default:
+			break; /* config has no opinion */
+		}
+
 		return FSMONITOR_REASON_REMOTE;
 	}
 
-- 
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
+/*
+ * Check if monitoring remote working directories is allowed.
+ *
+ * By default, monitoring remote working directories is
+ * disabled.  Users may override this behavior in enviroments where
+ * they have proper support.
+ */
+static int check_config_allowremote(struct repository *r)
+{
+	int allow;
+
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		return allow;
+
+	return -1; /* fsmonitor.allowremote not set */
+}
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Error if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
 /*
  * Remote working directories are problematic for FSMonitor.
  *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
  */
 static enum fsmonitor_reason check_remote(struct repository *r)
 {
+	int ret;
 	wchar_t wpath[MAX_PATH];
 	wchar_t wfullpath[MAX_PATH];
 	size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 		trace_printf_key(&trace_fsmonitor,
 				 "check_remote('%s') true",
 				 r->worktree);
+
+		ret = check_remote_protocol(wfullpath);
+		if (ret < 0)
+			return FSMONITOR_REASON_ERROR;
+
+		switch (check_config_allowremote(r)) {
+		case 0: /* config overrides and disables */
+			return FSMONITOR_REASON_REMOTE;
+		case 1: /* config overrides and enables */
+			return FSMONITOR_REASON_OK;
+		default:
+			break; /* config has no opinion */
+		}
+
 		return FSMONITOR_REASON_REMOTE;
 	}
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v2 2/4] fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 1/4] " Eric DeCosta via GitGitGadget
@ 2022-08-23 13:03   ` Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 3/4] Check working directory and Unix domain socket file for compatability edecosta via GitGitGadget
                     ` (2 subsequent siblings)
  4 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-23 13:03 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Follow-on to the work done to allow Windows to work against
network-mounted repos. Have macOS take advantage of the same
configuration option, 'fsmonitor.allowRemote' that was introduced for
Windows. Setting this option to true will override the default behavior
(erroring-out) when a network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for
IPC cannot be created in a network location; instead $HOME is used if
'fsmonitor.allowRemote' is true.

If $HOME is in a network location, allow the user to override this via
the 'fsmonitor.socketDir' configuration option. There the user can
specify any local directory for the location of the UDS file.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor-ipc.c      | 40 ++++++++++++++++++++++++--
 fsmonitor-ipc.h      |  6 ++++
 fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++--
 fsmonitor-settings.h |  4 +++
 4 files changed, 112 insertions(+), 5 deletions(-)

diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..1e3f0a6cf48 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -1,7 +1,6 @@
 #include "cache.h"
-#include "fsmonitor.h"
-#include "simple-ipc.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
+#ifdef WIN32
+	return fsmonitor_ipc__get_default_path();
+#else
+	char *retval;
+	SHA_CTX sha1ctx;
+	const char *git_dir;
+	const char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (fsm_settings__get_allow_remote(the_repository) < 1)
+		return fsmonitor_ipc__get_default_path();
+
+	git_dir = get_git_dir();
+	sock_dir = fsm_settings__get_socket_dir(the_repository);
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
+	SHA1_Final(hash, &sha1ctx);
+
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+	retval = interpolate_path(ipc_file.buf, 1);
+	if (!retval)
+		die(_("Invalid path: %s"), ipc_file.buf);
+	strbuf_release(&ipc_file);
+	return retval;
+#endif
+}
 
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..4d27223c2a6 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the default IPC named pipe or Unix domain
+ * socket.
+ */
+const char *fsmonitor_ipc__get_default_path(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..a15eeecebf4 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -10,7 +10,9 @@
 struct fsmonitor_settings {
 	enum fsmonitor_mode mode;
 	enum fsmonitor_reason reason;
+	int allow_remote;
 	char *hook_path;
+	char *sock_dir;
 };
 
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
 	CALLOC_ARRAY(s, 1);
 	s->mode = FSMONITOR_MODE_DISABLED;
 	s->reason = FSMONITOR_REASON_UNTESTED;
+	s->allow_remote = -1;
 
 	return s;
 }
@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)
 		fsm_settings__set_disabled(r);
 }
 
+int fsm_settings__get_allow_remote(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->allow_remote;
+}
+
+const char *fsm_settings__get_socket_dir(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->sock_dir;
+}
+
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 {
 	if (!r)
@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 	return r->settings.fsmonitor->mode;
 }
 
+
 const char *fsm_settings__get_hook_path(struct repository *r)
 {
 	if (!r)
@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 	return r->settings.fsmonitor->hook_path;
 }
 
+void fsm_settings__set_allow_remote(struct repository *r)
+{
+	int allow;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		r->settings.fsmonitor->allow_remote = allow;
+
+	return;
+}
+
+void fsm_settings__set_socket_dir(struct repository *r)
+{
+	const char *path;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+
+	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
+		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
+		r->settings.fsmonitor->sock_dir = strdup(path);
+	}
+
+	return;
+}
+
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..2de54c85e94 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v2 3/4] Check working directory and Unix domain socket file for compatability
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 1/4] " Eric DeCosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 2/4] fsmonitor: macOS: " Eric DeCosta via GitGitGadget
@ 2022-08-23 13:03   ` edecosta via GitGitGadget
  2022-08-23 13:03   ` [PATCH v2 4/4] Minor refactoring and simplification of Windows settings checks edecosta via GitGitGadget
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  4 siblings, 0 replies; 157+ messages in thread
From: edecosta via GitGitGadget @ 2022-08-23 13:03 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, edecosta

From: edecosta <edecosta@mathworks.com>

Perform separate checks for the working directory and Unix domain socket
(UDS) file location. The working directory may be located on a
network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS
file may never be located on a network-mounted file system; additionally
it may not be located on FAT32 or NTFS file systems.

Signed-off-by: edecosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 1 file changed, 52 insertions(+), 20 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dc79538607f 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v2 4/4] Minor refactoring and simplification of Windows settings checks
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
                     ` (2 preceding siblings ...)
  2022-08-23 13:03   ` [PATCH v2 3/4] Check working directory and Unix domain socket file for compatability edecosta via GitGitGadget
@ 2022-08-23 13:03   ` edecosta via GitGitGadget
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  4 siblings, 0 replies; 157+ messages in thread
From: edecosta via GitGitGadget @ 2022-08-23 13:03 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, edecosta

From: edecosta <edecosta@mathworks.com>

Read the value of 'fsmonitor.allowRemote' from fsmonitor_settings
via fsm_settings__get_allow_remote getter. Remove check_config_allowremote.
Replace switch statement with a simpler if statement. Move the check
for 'fsmonitor.allowRemote' above the remote protocol check.

Signed-off-by: edecosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-win32.c | 31 ++++-----------------------
 1 file changed, 4 insertions(+), 27 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index e5ec5b0a9f7..34635e6c849 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -24,23 +24,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
 /*
  * Check remote working directory protocol.
  *
@@ -170,20 +153,14 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 				 "check_remote('%s') true",
 				 r->worktree);
 
+		if (fsm_settings__get_allow_remote(r) < 1)
+			return FSMONITOR_REASON_REMOTE;
+
 		ret = check_remote_protocol(wfullpath);
 		if (ret < 0)
 			return FSMONITOR_REASON_ERROR;
 
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
+		return FSMONITOR_REASON_OK;
 	}
 
 	return FSMONITOR_REASON_OK;
-- 
@@ -24,23 +24,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
 /*
  * Check remote working directory protocol.
  *
@@ -170,20 +153,14 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 				 "check_remote('%s') true",
 				 r->worktree);
 
+		if (fsm_settings__get_allow_remote(r) < 1)
+			return FSMONITOR_REASON_REMOTE;
+
 		ret = check_remote_protocol(wfullpath);
 		if (ret < 0)
 			return FSMONITOR_REASON_ERROR;
 
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
+		return FSMONITOR_REASON_OK;
 	}
 
 	return FSMONITOR_REASON_OK;
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-22 13:22   ` Johannes Schindelin
  2022-08-22 16:07     ` Junio C Hamano
@ 2022-08-23 13:51     ` Jeff Hostetler
  2022-08-24 15:45       ` Eric DeCosta
  1 sibling, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-08-23 13:51 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, Eric DeCosta
  Cc: Johannes Schindelin, Junio C Hamano, git



On 8/22/22 9:22 AM, Johannes Schindelin wrote:
> Hi Junio,
> 
> On Sat, 20 Aug 2022, Junio C Hamano wrote:
> 
>>> As a first step towards enabling fsmonitor to work against
>>> network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
>>> was introduced for Windows. Setting this option to true will override
>>> the default behavior (erroring-out) when a network-mounted repo is
>>> detected by fsmonitor. In order for macOS to have parity with Windows,
>>> the same option is now introduced for macOS.
>>
>> With this merged in, recent CI runs for 'seen'
>>
>> e.g. https://github.com/git/git/actions/runs/2892889122
>>
>> seems to break macOS jobs, letting them hog CPU forever and exceed
>> 6hr or whatever the limit is.
>>
>> As an experiment I pushed out 'seen' but without this commit (not
>> the entire topic is excluded, the Windows one is still included).
>> As there is nothing specific to macOS between 'next' and 'seen',
>> macOS jobs seem to pass, which is not very surprising.
>>
>> https://github.com/git/git/actions/runs/2896207171
>>
>> As the patch collected some review comments, I've already marked it
>> in the "What's cooking" draft as expecting a reroll of that step;
>> until that happens, let's keep it out of 'seen'.
> 
> It makes sense to keep it out of `seen`, and at the same time I would like
> to encourage Eric to investigate what causes those time-outs.
> 
> When toggling timestamps (click on the wheel on the upper right) at
> https://github.com/git/git/runs/7927812510?check_suite_focus=true#step:4:1774,
> it can be seen that at close to 1am, t9903 finished, but then nothing
> happens until twenty past 6am.
> 
> I've downloaded the raw logs (also available via the wheel on the upper
> right) to find out which test timed out:
> 
> 	$ diff -u \
> 	  <(sed -n 's/.*\] \(t[0-9][^ ]*\).*/\1/p' <~/Downloads/17 | sort) \
> 	  <(git ls-tree upstream/seen:t | cut -c 54- | grep '^t[0-9].*-.*sh$')
> 
> 	--- /dev/fd/63  2022-08-22 14:56:05.510269527 +0200
> 	+++ /dev/fd/62  2022-08-22 14:56:05.510269527 +0200
> 	@@ -794,6 +794,7 @@
> 	 t7524-commit-summary.sh
> 	 t7525-status-rename.sh
> 	 t7526-commit-pathspec-file.sh
> 	+t7527-builtin-fsmonitor.sh
> 	 t7528-signed-commit-ssh.sh
> 	 t7600-merge.sh
> 	 t7601-merge-pull-config.sh
> 	@@ -945,6 +946,7 @@
> 	 t9812-git-p4-wildcards.sh
> 	 t9813-git-p4-preserve-users.sh
> 	 t9814-git-p4-rename.sh
> 	+t9815-git-p4-submit-fail.sh
> 	 t9816-git-p4-locked.sh
> 	 t9817-git-p4-exclude.sh
> 	 t9818-git-p4-block.sh
> 	@@ -964,5 +966,8 @@
> 	 t9832-unshelve.sh
> 	 t9833-errors.sh
> 	 t9834-git-p4-file-dir-bug.sh
> 	+t9835-git-p4-metadata-encoding-python2.sh
> 	+t9836-git-p4-metadata-encoding-python3.sh
> 	 t9901-git-web--browse.sh
> 	+t9902-completion.sh
> 	 t9903-bash-prompt.sh
> 
> I have no idea what's up with t98* and t9902, but I would bet that they
> were somehow "swallowed" by `prove` being terminated, and that the
> actual test script that times out is t7527.
> 
> Eric: To investigate, you will want to reproduce the problem on a macOS
> machine. If you have none available, you could create a temporary branch,
> heavily edit the CI definition, and push it to GitHub. And by heavy edits
> I mean something like this:
> 
> - Remove all non-macOS jobs from `.github/workflows/main.yml` (that means
>    removing all but the `regular` job, removing all but at least one
>    `macos` matrix entry, and removing the the `needs: ci-config` and
>    corresponding `if:` line.
> 
> - Edit `t/Makefile` to define `T = t7527-builtin-fsmonitor.sh` instead of
>    running all the tests.
> 
> - Edit `.github/workflows/main.yml` so that the step that causes the
>    time-out has a chance of timing out much sooner (and the subsequent
>    steps then have a chance to upload the relevant logs):
>    https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes

I would also set GIT_TRACE_FSMONITOR and GIT_TRACE2_PERF (on both daemon
and client sides of the tests) and capture the logs and try to figure
out what is happening.

I suspect that this testing should wait until you redo the patch to
remove the tmp file stuff and just move the socket into $HOME as we
talked about earlier.

Jeff


^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-23 13:03 ` [PATCH v2 0/4] " Eric DeCosta via GitGitGadget
                     ` (3 preceding siblings ...)
  2022-08-23 13:03   ` [PATCH v2 4/4] Minor refactoring and simplification of Windows settings checks edecosta via GitGitGadget
@ 2022-08-23 18:55   ` Eric DeCosta via GitGitGadget
  2022-08-23 18:55     ` [PATCH v3 1/2] fsmonitor: macOS: " Eric DeCosta via GitGitGadget
                       ` (3 more replies)
  4 siblings, 4 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-23 18:55 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta

cc: Johannes Schindelin Johannes.Schindelin@gmx.de cc: Jeff Hostetler
git@jeffhostetler.com cc: Eric Sunshine sunshine@sunshineco.com cc: Torsten
Bögershausen tboegi@web.de

Eric DeCosta (1):
  fsmonitor: macOS: allow fsmonitor to run against network-mounted repos

edecosta (1):
  Check working directory and Unix domain socket file for compatability

 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 fsmonitor-ipc.c                        | 40 ++++++++++++--
 fsmonitor-ipc.h                        |  6 +++
 fsmonitor-settings.c                   | 67 +++++++++++++++++++++++-
 fsmonitor-settings.h                   |  4 ++
 5 files changed, 164 insertions(+), 25 deletions(-)


base-commit: 795ea8776befc95ea2becd8020c7a284677b4161
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v2:

 1:  40ce21e85c9 < -:  ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos
 2:  46b4efd96cc = 1:  cd16d8bb3d6 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
 3:  9b128a98149 = 2:  f977d140afa Check working directory and Unix domain socket file for compatability
 4:  15c965801f8 < -:  ----------- Minor refactoring and simplification of Windows settings checks

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v3 1/2] fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
@ 2022-08-23 18:55     ` Eric DeCosta via GitGitGadget
  2022-08-23 18:55     ` [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability edecosta via GitGitGadget
                       ` (2 subsequent siblings)
  3 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-23 18:55 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Follow-on to the work done to allow Windows to work against
network-mounted repos. Have macOS take advantage of the same
configuration option, 'fsmonitor.allowRemote' that was introduced for
Windows. Setting this option to true will override the default behavior
(erroring-out) when a network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for
IPC cannot be created in a network location; instead $HOME is used if
'fsmonitor.allowRemote' is true.

If $HOME is in a network location, allow the user to override this via
the 'fsmonitor.socketDir' configuration option. There the user can
specify any local directory for the location of the UDS file.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor-ipc.c      | 40 ++++++++++++++++++++++++--
 fsmonitor-ipc.h      |  6 ++++
 fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++--
 fsmonitor-settings.h |  4 +++
 4 files changed, 112 insertions(+), 5 deletions(-)

diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..1e3f0a6cf48 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -1,7 +1,6 @@
 #include "cache.h"
-#include "fsmonitor.h"
-#include "simple-ipc.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
+#ifdef WIN32
+	return fsmonitor_ipc__get_default_path();
+#else
+	char *retval;
+	SHA_CTX sha1ctx;
+	const char *git_dir;
+	const char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (fsm_settings__get_allow_remote(the_repository) < 1)
+		return fsmonitor_ipc__get_default_path();
+
+	git_dir = get_git_dir();
+	sock_dir = fsm_settings__get_socket_dir(the_repository);
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
+	SHA1_Final(hash, &sha1ctx);
+
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+	retval = interpolate_path(ipc_file.buf, 1);
+	if (!retval)
+		die(_("Invalid path: %s"), ipc_file.buf);
+	strbuf_release(&ipc_file);
+	return retval;
+#endif
+}
 
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..4d27223c2a6 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the default IPC named pipe or Unix domain
+ * socket.
+ */
+const char *fsmonitor_ipc__get_default_path(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..a15eeecebf4 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -10,7 +10,9 @@
 struct fsmonitor_settings {
 	enum fsmonitor_mode mode;
 	enum fsmonitor_reason reason;
+	int allow_remote;
 	char *hook_path;
+	char *sock_dir;
 };
 
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
 	CALLOC_ARRAY(s, 1);
 	s->mode = FSMONITOR_MODE_DISABLED;
 	s->reason = FSMONITOR_REASON_UNTESTED;
+	s->allow_remote = -1;
 
 	return s;
 }
@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)
 		fsm_settings__set_disabled(r);
 }
 
+int fsm_settings__get_allow_remote(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->allow_remote;
+}
+
+const char *fsm_settings__get_socket_dir(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->sock_dir;
+}
+
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 {
 	if (!r)
@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 	return r->settings.fsmonitor->mode;
 }
 
+
 const char *fsm_settings__get_hook_path(struct repository *r)
 {
 	if (!r)
@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 	return r->settings.fsmonitor->hook_path;
 }
 
+void fsm_settings__set_allow_remote(struct repository *r)
+{
+	int allow;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		r->settings.fsmonitor->allow_remote = allow;
+
+	return;
+}
+
+void fsm_settings__set_socket_dir(struct repository *r)
+{
+	const char *path;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+
+	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
+		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
+		r->settings.fsmonitor->sock_dir = strdup(path);
+	}
+
+	return;
+}
+
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..2de54c85e94 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-08-23 18:55     ` [PATCH v3 1/2] fsmonitor: macOS: " Eric DeCosta via GitGitGadget
@ 2022-08-23 18:55     ` edecosta via GitGitGadget
  2022-08-24 20:31       ` Junio C Hamano
  2022-08-24 16:46     ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
  3 siblings, 1 reply; 157+ messages in thread
From: edecosta via GitGitGadget @ 2022-08-23 18:55 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, edecosta

From: edecosta <edecosta@mathworks.com>

Perform separate checks for the working directory and Unix domain socket
(UDS) file location. The working directory may be located on a
network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS
file may never be located on a network-mounted file system; additionally
it may not be located on FAT32 or NTFS file systems.

Signed-off-by: edecosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 1 file changed, 52 insertions(+), 20 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dc79538607f 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* RE: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-23 13:51     ` Jeff Hostetler
@ 2022-08-24 15:45       ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-08-24 15:45 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget
  Cc: Johannes Schindelin, Junio C Hamano, git

> I would also set GIT_TRACE_FSMONITOR and GIT_TRACE2_PERF (on both daemon
> and client sides of the tests) and capture the logs and try to figure
> out what is happening.
>
> I suspect that this testing should wait until you redo the patch to
> remove the tmp file stuff and just move the socket into $HOME as we
> talked about earlier.
>
> Jeff

All tests are passing now with the new patch. By default the socket is written into
the original location (.git directory) unless 'fsmonitor.allowRemote' is true. Only
then is $HOME used and if $HOME proves unsuitable the user can override that
by setting 'fsmonitor.sockerDir' to some valid, local directory.

-Eric

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-08-23 18:55     ` [PATCH v3 1/2] fsmonitor: macOS: " Eric DeCosta via GitGitGadget
  2022-08-23 18:55     ` [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability edecosta via GitGitGadget
@ 2022-08-24 16:46     ` Junio C Hamano
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
  3 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-08-24 16:46 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> cc: Johannes Schindelin Johannes.Schindelin@gmx.de cc: Jeff Hostetler
> git@jeffhostetler.com cc: Eric Sunshine sunshine@sunshineco.com cc: Torsten
> Bögershausen tboegi@web.de
>
> Eric DeCosta (1):
>   fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
>
> edecosta (1):
>   Check working directory and Unix domain socket file for compatability

Please describe why v3 exists (iow, what was wrong in v2 that v3
improves) in the cover letter.  Especially when you send out two
revisions in less than 12 hours, it gets disorienting at the
receiving end.

Thanks.

>  compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
>  fsmonitor-ipc.c                        | 40 ++++++++++++--
>  fsmonitor-ipc.h                        |  6 +++
>  fsmonitor-settings.c                   | 67 +++++++++++++++++++++++-
>  fsmonitor-settings.h                   |  4 ++
>  5 files changed, 164 insertions(+), 25 deletions(-)
>
>
> base-commit: 795ea8776befc95ea2becd8020c7a284677b4161
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v3
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v3
> Pull-Request: https://github.com/gitgitgadget/git/pull/1326
>
> Range-diff vs v2:
>
>  1:  40ce21e85c9 < -:  ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos
>  2:  46b4efd96cc = 1:  cd16d8bb3d6 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
>  3:  9b128a98149 = 2:  f977d140afa Check working directory and Unix domain socket file for compatability
>  4:  15c965801f8 < -:  ----------- Minor refactoring and simplification of Windows settings checks

^ permalink raw reply	[flat|nested] 157+ messages in thread

* (no subject)
  2022-08-23 18:55     ` [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability edecosta via GitGitGadget
@ 2022-08-24 20:31       ` Junio C Hamano
  0 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-08-24 20:31 UTC (permalink / raw)
  To: edecosta via GitGitGadget; +Cc: git, Eric DeCosta

"edecosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: edecosta <edecosta@mathworks.com>

Do you have some specific reason to avoid using

    Eric DeCosta <edecosta@mathworks.com>

to identify yourself?  Otherwise, we'd prefer to see it that way.

> Subject: Re: [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability

This project is wider than just its fsmonitor part, so please make
sure to give your change a title that will not be lost in the noise
when it is shown together with other patfches in the output from
"git shortlog --no-merges".  I haven't spent more than 30 seconds so
you may be able to summarize it better, but perhaps

fsmonitor: ensure filesystem and unix socket are usable by fsmonitor

or something?  If I am reading the early part of the patch
correctly, the value of this change primarily comes from the fact
that it separates (1) filesystems that are inherently incompatible
(e.g. cannot create sockets) and (2) filesystems that pretend to
successfully create sockets but the sockets are unusable, so a title
with "separately" somewhere might convey its essence more clearly.
I dunno.

The patch itself looked sensible (I didn't look for typos in
constants etc., though).

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v4 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-23 18:55   ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                       ` (2 preceding siblings ...)
  2022-08-24 16:46     ` [PATCH v3 0/2] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
@ 2022-08-31 16:09     ` Eric DeCosta via GitGitGadget
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
                         ` (4 more replies)
  3 siblings, 5 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-31 16:09 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta

cc: Johannes Schindelin Johannes.Schindelin@gmx.de cc: Jeff Hostetler
git@jeffhostetler.com cc: Eric Sunshine sunshine@sunshineco.com cc: Torsten
Bögershausen tboegi@web.de

Allow fsmonitor to run against network-mounted repos on macOS.

There are four parts to this effort:

 1. Introduce two new configuration options
    
    fsmonitor.allowRemote - setting this to true overrides fsmonitor's
    default behavior of erroring out when enountering network file systems.
    Additionly, when true, the Unix domain socket (UDS) file used for IPC is
    located in $HOME rather than in the .git directory.
    
    fsmonitor.socketDir - allows for the UDS file to be located anywhere the
    user chooses rather $HOME.

 2. Using the values of above configuration options, locate the UDS file in
    the desired location with a unique name based on the SHA1 of the path of
    the .git folder.

 3. Ensure that both the working directory (.git directory) and the UDS file
    location are compatible with fsmonitor

 4. Normalize the paths returned by FSEvents to the real path for each
    affected file or directory

Eric DeCosta (4):
  fsmonitor: add two new config options, allowRemote and socketDir
  fsmonitor: generate unique Unix socket file name in the desired
    location
  fsmonitor: ensure filesystem and unix socket filesystem are compatible
  fsmonitor: normalize FSEvents event paths to the real path

 compat/fsmonitor/fsm-listen-darwin.c   | 11 ++--
 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 fsmonitor-ipc.c                        | 40 ++++++++++++--
 fsmonitor-ipc.h                        |  6 +++
 fsmonitor-settings.c                   | 67 +++++++++++++++++++++++-
 fsmonitor-settings.h                   |  4 ++
 6 files changed, 172 insertions(+), 28 deletions(-)


base-commit: 795ea8776befc95ea2becd8020c7a284677b4161
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v3:

 1:  cd16d8bb3d6 ! 1:  836a791e6b7 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
     @@ Metadata
      Author: Eric DeCosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
     +    fsmonitor: add two new config options, allowRemote and socketDir
      
     -    Follow-on to the work done to allow Windows to work against
     -    network-mounted repos. Have macOS take advantage of the same
     -    configuration option, 'fsmonitor.allowRemote' that was introduced for
     -    Windows. Setting this option to true will override the default behavior
     -    (erroring-out) when a network-mounted repo is detected by fsmonitor.
     +    Introduce two new configuration options
      
     -    The added wrinkle being that the Unix domain socket (UDS) file used for
     -    IPC cannot be created in a network location; instead $HOME is used if
     -    'fsmonitor.allowRemote' is true.
     +       fsmonitor.allowRemote - setting this to true overrides fsmonitor's
     +       default behavior of erroring out when enountering network file
     +       systems. Additionly, when true, the Unix domain socket (UDS) file
     +       used for IPC is located in $HOME rather than in the .git directory.
      
     -    If $HOME is in a network location, allow the user to override this via
     -    the 'fsmonitor.socketDir' configuration option. There the user can
     -    specify any local directory for the location of the UDS file.
     +       fsmonitor.socketDir - allows for the UDS file to be located
     +       anywhere the user chooses rather $HOME.
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     - ## fsmonitor-ipc.c ##
     -@@
     - #include "cache.h"
     --#include "fsmonitor.h"
     --#include "simple-ipc.h"
     - #include "fsmonitor-ipc.h"
     -+#include "fsmonitor-settings.h"
     - #include "run-command.h"
     - #include "strbuf.h"
     - #include "trace2.h"
     -@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
     - 	return 1;
     - }
     - 
     --GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     -+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
     -+
     -+const char *fsmonitor_ipc__get_path(void)
     -+{
     -+#ifdef WIN32
     -+	return fsmonitor_ipc__get_default_path();
     -+#else
     -+	char *retval;
     -+	SHA_CTX sha1ctx;
     -+	const char *git_dir;
     -+	const char *sock_dir;
     -+	struct strbuf ipc_file = STRBUF_INIT;
     -+	unsigned char hash[SHA_DIGEST_LENGTH];
     -+
     -+	if (fsm_settings__get_allow_remote(the_repository) < 1)
     -+		return fsmonitor_ipc__get_default_path();
     -+
     -+	git_dir = get_git_dir();
     -+	sock_dir = fsm_settings__get_socket_dir(the_repository);
     -+
     -+	SHA1_Init(&sha1ctx);
     -+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
     -+	SHA1_Final(hash, &sha1ctx);
     -+
     -+	if (sock_dir && *sock_dir)
     -+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
     -+					sock_dir, hash_to_hex(hash));
     -+	else
     -+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
     -+	retval = interpolate_path(ipc_file.buf, 1);
     -+	if (!retval)
     -+		die(_("Invalid path: %s"), ipc_file.buf);
     -+	strbuf_release(&ipc_file);
     -+	return retval;
     -+#endif
     -+}
     - 
     - enum ipc_active_state fsmonitor_ipc__get_state(void)
     - {
     -
     - ## fsmonitor-ipc.h ##
     -@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
     -  */
     - const char *fsmonitor_ipc__get_path(void);
     - 
     -+/*
     -+ * Returns the pathname to the default IPC named pipe or Unix domain
     -+ * socket.
     -+ */
     -+const char *fsmonitor_ipc__get_default_path(void);
     -+
     - /*
     -  * Try to determine whether there is a `git-fsmonitor--daemon` process
     -  * listening on the IPC pipe/socket.
     -
       ## fsmonitor-settings.c ##
      @@
       struct fsmonitor_settings {
 -:  ----------- > 2:  2cb026a6317 fsmonitor: generate unique Unix socket file name in the desired location
 2:  f977d140afa ! 3:  a3110f1e25a Check working directory and Unix domain socket file for compatability
     @@
       ## Metadata ##
     -Author: edecosta <edecosta@mathworks.com>
     +Author: Eric DeCosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    Check working directory and Unix domain socket file for compatability
     +    fsmonitor: ensure filesystem and unix socket filesystem are compatible
      
          Perform separate checks for the working directory and Unix domain socket
          (UDS) file location. The working directory may be located on a
     @@ Commit message
          file may never be located on a network-mounted file system; additionally
          it may not be located on FAT32 or NTFS file systems.
      
     -    Signed-off-by: edecosta <edecosta@mathworks.com>
     +    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
       ## compat/fsmonitor/fsm-settings-darwin.c ##
      @@
 -:  ----------- > 4:  56cabf3be3b fsmonitor: normalize FSEvents event paths to the real path

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
@ 2022-08-31 16:09       ` Eric DeCosta via GitGitGadget
  2022-08-31 19:41         ` Ævar Arnfjörð Bjarmason
                           ` (4 more replies)
  2022-08-31 16:09       ` [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location Eric DeCosta via GitGitGadget
                         ` (3 subsequent siblings)
  4 siblings, 5 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-31 16:09 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Introduce two new configuration options

   fsmonitor.allowRemote - setting this to true overrides fsmonitor's
   default behavior of erroring out when enountering network file
   systems. Additionly, when true, the Unix domain socket (UDS) file
   used for IPC is located in $HOME rather than in the .git directory.

   fsmonitor.socketDir - allows for the UDS file to be located
   anywhere the user chooses rather $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++--
 fsmonitor-settings.h |  4 +++
 2 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..a15eeecebf4 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -10,7 +10,9 @@
 struct fsmonitor_settings {
 	enum fsmonitor_mode mode;
 	enum fsmonitor_reason reason;
+	int allow_remote;
 	char *hook_path;
+	char *sock_dir;
 };
 
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
 	CALLOC_ARRAY(s, 1);
 	s->mode = FSMONITOR_MODE_DISABLED;
 	s->reason = FSMONITOR_REASON_UNTESTED;
+	s->allow_remote = -1;
 
 	return s;
 }
@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)
 		fsm_settings__set_disabled(r);
 }
 
+int fsm_settings__get_allow_remote(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->allow_remote;
+}
+
+const char *fsm_settings__get_socket_dir(struct repository *r)
+{
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		lookup_fsmonitor_settings(r);
+
+	return r->settings.fsmonitor->sock_dir;
+}
+
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 {
 	if (!r)
@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
 	return r->settings.fsmonitor->mode;
 }
 
+
 const char *fsm_settings__get_hook_path(struct repository *r)
 {
 	if (!r)
@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 	return r->settings.fsmonitor->hook_path;
 }
 
+void fsm_settings__set_allow_remote(struct repository *r)
+{
+	int allow;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		r->settings.fsmonitor->allow_remote = allow;
+
+	return;
+}
+
+void fsm_settings__set_socket_dir(struct repository *r)
+{
+	const char *path;
+
+	if (!r)
+		r = the_repository;
+	if (!r->settings.fsmonitor)
+		r->settings.fsmonitor = alloc_settings();
+
+	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
+		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
+		r->settings.fsmonitor->sock_dir = strdup(path);
+	}
+
+	return;
+}
+
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason;
+
+	fsm_settings__set_allow_remote(r);
+	fsm_settings__set_socket_dir(r);
+	reason = check_for_incompatible(r);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..2de54c85e94 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
@@ -23,12 +23,16 @@ enum fsmonitor_reason {
 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
 };
 
+void fsm_settings__set_allow_remote(struct repository *r);
+void fsm_settings__set_socket_dir(struct repository *r);
 void fsm_settings__set_ipc(struct repository *r);
 void fsm_settings__set_hook(struct repository *r, const char *path);
 void fsm_settings__set_disabled(struct repository *r);
 void fsm_settings__set_incompatible(struct repository *r,
 				    enum fsmonitor_reason reason);
 
+int fsm_settings__get_allow_remote(struct repository *r);
+const char *fsm_settings__get_socket_dir(struct repository *r);
 enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
 const char *fsm_settings__get_hook_path(struct repository *r);
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
@ 2022-08-31 16:09       ` Eric DeCosta via GitGitGadget
  2022-08-31 19:49         ` Ævar Arnfjörð Bjarmason
  2022-08-31 20:11         ` Junio C Hamano
  2022-08-31 16:09       ` [PATCH v4 3/4] fsmonitor: ensure filesystem and unix socket filesystem are compatible Eric DeCosta via GitGitGadget
                         ` (2 subsequent siblings)
  4 siblings, 2 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-31 16:09 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir
locate the Unix domain socket file in the desired location (either
the .git directory, $HOME, or fsmonitor.socketDir). If the location
is other than the .git directory, generate a unique file name based
on the SHA1 has of the path to the .git directory.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor-ipc.c | 40 +++++++++++++++++++++++++++++++++++++---
 fsmonitor-ipc.h |  6 ++++++
 2 files changed, 43 insertions(+), 3 deletions(-)

diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..1e3f0a6cf48 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -1,7 +1,6 @@
 #include "cache.h"
-#include "fsmonitor.h"
-#include "simple-ipc.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
+#ifdef WIN32
+	return fsmonitor_ipc__get_default_path();
+#else
+	char *retval;
+	SHA_CTX sha1ctx;
+	const char *git_dir;
+	const char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (fsm_settings__get_allow_remote(the_repository) < 1)
+		return fsmonitor_ipc__get_default_path();
+
+	git_dir = get_git_dir();
+	sock_dir = fsm_settings__get_socket_dir(the_repository);
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
+	SHA1_Final(hash, &sha1ctx);
+
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+	retval = interpolate_path(ipc_file.buf, 1);
+	if (!retval)
+		die(_("Invalid path: %s"), ipc_file.buf);
+	strbuf_release(&ipc_file);
+	return retval;
+#endif
+}
 
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..4d27223c2a6 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the default IPC named pipe or Unix domain
+ * socket.
+ */
+const char *fsmonitor_ipc__get_default_path(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.
-- 
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);
  */
 const char *fsmonitor_ipc__get_path(void);
 
+/*
+ * Returns the pathname to the default IPC named pipe or Unix domain
+ * socket.
+ */
+const char *fsmonitor_ipc__get_default_path(void);
+
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
  * listening on the IPC pipe/socket.
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v4 3/4] fsmonitor: ensure filesystem and unix socket filesystem are compatible
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
  2022-08-31 16:09       ` [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location Eric DeCosta via GitGitGadget
@ 2022-08-31 16:09       ` Eric DeCosta via GitGitGadget
  2022-08-31 16:09       ` [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path Eric DeCosta via GitGitGadget
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  4 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-31 16:09 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Perform separate checks for the working directory and Unix domain socket
(UDS) file location. The working directory may be located on a
network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS
file may never be located on a network-mounted file system; additionally
it may not be located on FAT32 or NTFS file systems.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++-------
 1 file changed, 52 insertions(+), 20 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dc79538607f 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
@@ -3,11 +3,12 @@
 #include "repository.h"
 #include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
 #include <sys/param.h>
 #include <sys/mount.h>
 
 /*
- * [1] Remote working directories are problematic for FSMonitor.
+ * Remote working directories are problematic for FSMonitor.
  *
  * The underlying file system on the server machine and/or the remote
  * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@
  * In theory, the above issues need to be addressed whether we are
  * using the Hook or IPC API.
  *
- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory.  If the working directory is remote,
- * then the socket will be created on the remote file system.  This
- * can fail if the remote file system does not support UDS file types
- * (e.g. smbfs to a Windows server) or if the remote kernel does not
- * allow a non-local process to bind() the socket.  (These problems
- * could be fixed by moving the UDS out of the .git directory and to a
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
- *
  * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
- * directory for IPC.  These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+ * incompatible unless fsmonitor.allowRemote is true.
  *
  */
 static enum fsmonitor_reason check_volume(struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)
 			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
 			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
 
+	if (!(fs.f_flags & MNT_LOCAL)
+		&& (fsm_settings__get_allow_remote(r) < 1))
+			return FSMONITOR_REASON_REMOTE;
+
+	return FSMONITOR_REASON_OK;
+}
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
+ * for the IPC in the .git directory by default or $HOME if
+ * fsmonitor.allowRemote is true.  If the directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket.
+ *
+ * Therefore remote UDS locations are marked as incompatible.
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * These Windows drive formats do not support Unix domain sockets, so
+ * mark them as incompatible for the location of the UDS file.
+ *
+ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+	struct statfs fs;
+	struct strbuf path = STRBUF_INIT;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
+
+	if (statfs(dirname(path.buf), &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path.buf, strerror(saved_errno));
+		errno = saved_errno;
+		strbuf_release(&path);
+		return FSMONITOR_REASON_ERROR;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
+
 	if (!(fs.f_flags & MNT_LOCAL))
 		return FSMONITOR_REASON_REMOTE;
 
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
+	reason = check_uds_volume();
+	if (reason != FSMONITOR_REASON_OK)
+		return reason;
+
 	return FSMONITOR_REASON_OK;
 }
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
                         ` (2 preceding siblings ...)
  2022-08-31 16:09       ` [PATCH v4 3/4] fsmonitor: ensure filesystem and unix socket filesystem are compatible Eric DeCosta via GitGitGadget
@ 2022-08-31 16:09       ` Eric DeCosta via GitGitGadget
  2022-08-31 19:37         ` Ævar Arnfjörð Bjarmason
  2022-09-01 20:05         ` Jeff Hostetler
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  4 siblings, 2 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-08-31 16:09 UTC (permalink / raw)
  To: git; +Cc: Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Consider the following network working directory that is mounted under
/System/Volumes/Data:

/network/working/directory

The git working directory path is:

/System/Volumes/Data/network/working/directory

The paths reported by FSEvents always start with /network. fsmonitor
expects paths to be under the working directory; therefore it
fails to match /network/... and ignores the change.

Change things such that if fsmonitor.allowRemote is true that the
paths reported via FSEevents are normalized to the real path.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..2ed828649ff 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-settings.h"
 
 struct fsm_listen_data
 {
@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)
 	free(composed);
 }
 
-
 static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			     void *ctx,
 			     size_t num_of_events,
@@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		if (fsm_settings__get_allow_remote(the_repository) > 0) {
+			strbuf_reset(&tmp);
+			strbuf_realpath_forgiving(&tmp, paths[k], 0);
+			path_k = tmp.buf;
+		} else
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -237,6 +242,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
+			batch = NULL;
 			string_list_clear(&cookie_list, 0);
 
 			/*
@@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 
 		case IS_WORKDIR_PATH:
 			/* try to queue normal pathnames */
-
 			if (trace_pass_fl(&trace_fsmonitor))
 				log_flags_set(path_k, event_flags[k]);
 
-- 
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-settings.h"
 
 struct fsm_listen_data
 {
@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)
 	free(composed);
 }
 
-
 static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			     void *ctx,
 			     size_t num_of_events,
@@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		if (fsm_settings__get_allow_remote(the_repository) > 0) {
+			strbuf_reset(&tmp);
+			strbuf_realpath_forgiving(&tmp, paths[k], 0);
+			path_k = tmp.buf;
+		} else
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -237,6 +242,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
+			batch = NULL;
 			string_list_clear(&cookie_list, 0);
 
 			/*
@@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 
 		case IS_WORKDIR_PATH:
 			/* try to queue normal pathnames */
-
 			if (trace_pass_fl(&trace_fsmonitor))
 				log_flags_set(path_k, event_flags[k]);
 
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-08-31 16:09       ` [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path Eric DeCosta via GitGitGadget
@ 2022-08-31 19:37         ` Ævar Arnfjörð Bjarmason
  2022-09-01 20:05         ` Jeff Hostetler
  1 sibling, 0 replies; 157+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-08-31 19:37 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta


On Wed, Aug 31 2022, Eric DeCosta via GitGitGadget wrote:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> Consider the following network working directory that is mounted under
> /System/Volumes/Data:
>
> /network/working/directory
>
> The git working directory path is:
>
> /System/Volumes/Data/network/working/directory
>
> The paths reported by FSEvents always start with /network. fsmonitor
> expects paths to be under the working directory; therefore it
> fails to match /network/... and ignores the change.
>
> Change things such that if fsmonitor.allowRemote is true that the
> paths reported via FSEevents are normalized to the real path.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
>  compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
> index 8e208e8289e..2ed828649ff 100644
> --- a/compat/fsmonitor/fsm-listen-darwin.c
> +++ b/compat/fsmonitor/fsm-listen-darwin.c
> @@ -26,6 +26,7 @@
>  #include "fsmonitor.h"
>  #include "fsm-listen.h"
>  #include "fsmonitor--daemon.h"
> +#include "fsmonitor-settings.h"
>  
>  struct fsm_listen_data
>  {
> @@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)
>  	free(composed);
>  }
>  
> -

Stray whitespace change, any one isn't much, but they add up (I saw
another one earlier in this topic).

>  static void fsevent_callback(ConstFSEventStreamRef streamRef,
>  			     void *ctx,
>  			     size_t num_of_events,
> @@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
>  		/*
>  		 * On Mac, we receive an array of absolute paths.
>  		 */
> -		path_k = paths[k];
> +		if (fsm_settings__get_allow_remote(the_repository) > 0) {
> +			strbuf_reset(&tmp);
> +			strbuf_realpath_forgiving(&tmp, paths[k], 0);
> +			path_k = tmp.buf;
> +		} else
> +			path_k = paths[k];

Style: This else should have braces if any if/else arm does, see
CodingGuidelines.

> [...]
> @@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
>  
>  		case IS_WORKDIR_PATH:
>  			/* try to queue normal pathnames */
> -

More stray whitespace

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
@ 2022-08-31 19:41         ` Ævar Arnfjörð Bjarmason
  2022-08-31 20:04         ` Junio C Hamano
                           ` (3 subsequent siblings)
  4 siblings, 0 replies; 157+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-08-31 19:41 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta


On Wed, Aug 31 2022, Eric DeCosta via GitGitGadget wrote:

> From: Eric DeCosta <edecosta@mathworks.com>
> [...]
>  	enum fsmonitor_reason reason;
> +	int allow_remote;
>  	char *hook_path;
> +	char *sock_dir;
>  };

Any reason we couldn't just add this to "struct repo_settings" and ...

> +int fsm_settings__get_allow_remote(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->allow_remote;
> +}
> +
> +const char *fsm_settings__get_socket_dir(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->sock_dir;
> +}
> +

...instead of this whole ceremony...

> +void fsm_settings__set_allow_remote(struct repository *r)
> +{
> +	int allow;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
> +		r->settings.fsmonitor->allow_remote = allow;
> +
> +	return;
> +}

Just have a single repo_cfg_bool() line in prepare_repo_settings()
instead?

(There are some reasons for the "lazy" behavior of fsmonitor-settings.c,
but surely a simple boolean variable we can read on startup isn't it,
and we already paid the cost to do so with the configset...)


> +void fsm_settings__set_socket_dir(struct repository *r)
> +{
> +	const char *path;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +
> +	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
> +		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);

...maybe this socket dir stuff is the exception though.

> +		r->settings.fsmonitor->sock_dir = strdup(path);

Aren't you strdup()-ing an already strdup()'d string, and therefore
leaking memory? Also this should be xstrdup(), surely?

> +	}
> +
> +	return;
> +}
> +
>  void fsm_settings__set_ipc(struct repository *r)
>  {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);

This seems rather backwards, as odd as this API itself is already, isn't
the whole idea that after we call lookup_fsmonitor_settings() it will
have set() anything that's mandatory?

But maybe I haven't grokked it ... :)

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location
  2022-08-31 16:09       ` [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location Eric DeCosta via GitGitGadget
@ 2022-08-31 19:49         ` Ævar Arnfjörð Bjarmason
  2022-08-31 20:11         ` Junio C Hamano
  1 sibling, 0 replies; 157+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2022-08-31 19:49 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta


On Wed, Aug 31 2022, Eric DeCosta via GitGitGadget wrote:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir
> locate the Unix domain socket file in the desired location (either
> the .git directory, $HOME, or fsmonitor.socketDir). If the location
> is other than the .git directory, generate a unique file name based
> on the SHA1 has of the path to the .git directory.

Per:

	fsmonitor-ipc.h- * Returns the pathname to the IPC named pipe or Unix domain socket
	fsmonitor-ipc.h- * where a `git-fsmonitor--daemon` process will listen.  This is a
	fsmonitor-ipc.h- * per-worktree value.

> +	git_dir = get_git_dir();
> +	sock_dir = fsm_settings__get_socket_dir(the_repository);
> +
> +	SHA1_Init(&sha1ctx);
> +	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
> +	SHA1_Final(hash, &sha1ctx);
> +
> +	if (sock_dir && *sock_dir)
> +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> +					sock_dir, hash_to_hex(hash));
> +	else

But here we (from eyeballing this, maybe I've missed something):

 1. Get the path to the git dir
 2. SHA-1 hash that path, presumably to make it fixed size & get rid of
    slashes etc.
 3. Make that the IPC filename

Per the "per worktree" can't we just check if:

 * We have a .git/worktree/? If so derive the name from that.
 * We don't? Then we just have one? Stick it in in there? I.e. isn't the
   hash_to_hex() here redundant?

...

> +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));

... but not here, so is it just for this "else", but got carried over
above?

I think if we're creating a new global cookie file, and presumably
potentially a *lot* of them in the user's ~ we should at least
prominently document that somewhere.

But more generally couldn't this?:

 * Play nice with $HOME/.config/git/ etc, as is the usual convention these days on *nix
 * We would have to the equivalent of a "mkdir -p", but if we stick it
   in .config/git/fsmonitor-sockets or something we could have a nested
   path there that mirrors the path to the repo.

The latter can really help with debugging, you have this random
.git-fsmonitor-XXXXX file, where the XXXX is totally mysterious, until
you eventually find this code & discover it's a SHA-1 hash of some other
path...

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
  2022-08-31 19:41         ` Ævar Arnfjörð Bjarmason
@ 2022-08-31 20:04         ` Junio C Hamano
  2022-09-01  2:25           ` Ramsay Jones
  2022-09-01 17:53         ` Jeff Hostetler
                           ` (2 subsequent siblings)
  4 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-08-31 20:04 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> Introduce two new configuration options
>
>    fsmonitor.allowRemote - setting this to true overrides fsmonitor's
>    default behavior of erroring out when enountering network file
>    systems. Additionly, when true, the Unix domain socket (UDS) file
>    used for IPC is located in $HOME rather than in the .git directory.
>
>    fsmonitor.socketDir - allows for the UDS file to be located
>    anywhere the user chooses rather $HOME.

Before describing "what they do", please justify why we need to add
them.  The usual way to do so is to start with some observation of
the status quo, and describe the "gap" between what can be done with
the current system and what the users would want to do.  It might
further be necessary to justify why "would want to do" is worthwhile
to support.  I suspect that you can do all of the above in a couple
of paragraphs, and you'd succeed if the solution you chose would
fall out as a natural consequence in the mind of readers who follow
your line of thought by reading these introductory paragraphs.  And
then after the stage is set like so, the above description of what
you chose to implement as a solution should come.

>  struct fsmonitor_settings {
>  	enum fsmonitor_mode mode;
>  	enum fsmonitor_reason reason;
> +	int allow_remote;

I am debating myuself if a comment like

	int allow_remote; /* -1: undecided, 0: unallowed, 1: allowed */

is necessary (I know it would help if we had one; I am just
wondering if it is too obvious).

>  	char *hook_path;
> +	char *sock_dir;
>  };
>  
>  static enum fsmonitor_reason check_for_incompatible(struct repository *r)
> @@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
>  	CALLOC_ARRAY(s, 1);
>  	s->mode = FSMONITOR_MODE_DISABLED;
>  	s->reason = FSMONITOR_REASON_UNTESTED;
> +	s->allow_remote = -1;
>  
>  	return s;
>  }

OK.  socket_dir is naturally NULL at the start.

> @@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>  	return r->settings.fsmonitor->mode;
>  }
>  
> +
>  const char *fsm_settings__get_hook_path(struct repository *r)
>  {
>  	if (!r)

A noise hunk?

> @@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
> ...
> +void fsm_settings__set_socket_dir(struct repository *r)
> +{
> +	const char *path;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +
> +	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
> +		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
> +		r->settings.fsmonitor->sock_dir = strdup(path);

As we are overwriting it immediately, just calling free(), not
FREE_AND_NULL(), is more appropriate, isn't it?

> @@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
>  
>  void fsm_settings__set_hook(struct repository *r, const char *path)
>  {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);
>  
>  	if (reason != FSMONITOR_REASON_OK) {
>  		fsm_settings__set_incompatible(r, reason);
> diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
> index d9c2605197f..2de54c85e94 100644
> --- a/fsmonitor-settings.h
> +++ b/fsmonitor-settings.h
> @@ -23,12 +23,16 @@ enum fsmonitor_reason {
>  	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
>  };
>  
> +void fsm_settings__set_allow_remote(struct repository *r);
> +void fsm_settings__set_socket_dir(struct repository *r);

Do these two need to be extern?

I would imagine that implementations in compat/fsmonitor/* would
just call fsm_settings__set_hook() or __set_ipc() and that causes
these two to be called as part of the set-up sequence.  Do they need
to call these two directly?

If not, we probably should make them static.  I suspect that some
existing declarations in this header file fall into the same
category and may need to become static for the same reason, which
you can do as a preliminary clean-up patch, or post-clean-up after
the dust settles.  Regardless of which approach to take for existing
ones, we do not want to make it worse by adding new externally
visible names that do not have to be visible.

>  void fsm_settings__set_ipc(struct repository *r);
>  void fsm_settings__set_hook(struct repository *r, const char *path);
>  void fsm_settings__set_disabled(struct repository *r);
>  void fsm_settings__set_incompatible(struct repository *r,
>  				    enum fsmonitor_reason reason);
>  
> +int fsm_settings__get_allow_remote(struct repository *r);
> +const char *fsm_settings__get_socket_dir(struct repository *r);

On the other hand, these may need to be query-able by the
implementations.

>  enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
>  const char *fsm_settings__get_hook_path(struct repository *r);

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location
  2022-08-31 16:09       ` [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location Eric DeCosta via GitGitGadget
  2022-08-31 19:49         ` Ævar Arnfjörð Bjarmason
@ 2022-08-31 20:11         ` Junio C Hamano
  1 sibling, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-08-31 20:11 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +const char *fsmonitor_ipc__get_path(void)
> +{
> +#ifdef WIN32
> +	return fsmonitor_ipc__get_default_path();
> +#else

Hmph.

As compat/fsmonitor/ directory already sets up a good way to have
platform-specific implementation of a common API function, I think
this patch goes in a wrong direction by fighting it.

Shouldn't the rest of the function we see here be made to another 
implementation of fsmonitor_ipc__get_default_path() that is
specific to platforms, i.e. in compat/fsmonitor/$somenewfile.c,
that is used for !WIN32 case?

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 20:04         ` Junio C Hamano
@ 2022-09-01  2:25           ` Ramsay Jones
  0 siblings, 0 replies; 157+ messages in thread
From: Ramsay Jones @ 2022-09-01  2:25 UTC (permalink / raw)
  To: Junio C Hamano, Eric DeCosta via GitGitGadget; +Cc: git, Eric DeCosta



On 31/08/2022 21:04, Junio C Hamano wrote:
> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
[snip]
>> diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
>> index d9c2605197f..2de54c85e94 100644
>> --- a/fsmonitor-settings.h
>> +++ b/fsmonitor-settings.h
>> @@ -23,12 +23,16 @@ enum fsmonitor_reason {
>>  	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
>>  };
>>  
>> +void fsm_settings__set_allow_remote(struct repository *r);
>> +void fsm_settings__set_socket_dir(struct repository *r);
> 
> Do these two need to be extern?
> 

On 'seen' after building:
  ...
  $ ./static-check.pl >ssc
  $ diff nsc ssc
  3a4
  > bundle-uri.o	- for_all_bundles_in_list
  15d15
  < config.o	- git_config_from_file_with_options
  40a41,43
  > fsmonitor-settings.o	- fsm_settings__get_allow_remote
  > fsmonitor-settings.o	- fsm_settings__get_socket_dir
  > fsmonitor-settings.o	- fsm_settings__set_allow_remote
  44a48
  > fsmonitor-settings.o	- fsm_settings__set_socket_dir
  $ 

.. so probably not. ;-)

ATB,
Ramsay Jones



^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
  2022-08-31 19:41         ` Ævar Arnfjörð Bjarmason
  2022-08-31 20:04         ` Junio C Hamano
@ 2022-09-01 17:53         ` Jeff Hostetler
  2022-09-01 18:04         ` Jeff Hostetler
  2022-09-01 21:21         ` Jeff Hostetler
  4 siblings, 0 replies; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-01 17:53 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git; +Cc: Eric DeCosta



On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>
> 
> Introduce two new configuration options
> 
>     fsmonitor.allowRemote - setting this to true overrides fsmonitor's
>     default behavior of erroring out when enountering network file
>     systems. Additionly, when true, the Unix domain socket (UDS) file
>     used for IPC is located in $HOME rather than in the .git directory.
> 
>     fsmonitor.socketDir - allows for the UDS file to be located
>     anywhere the user chooses rather $HOME.
> 
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
>   fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++--
>   fsmonitor-settings.h |  4 +++
>   2 files changed, 69 insertions(+), 2 deletions(-)
> 
> diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
> index 464424a1e92..a15eeecebf4 100644
> --- a/fsmonitor-settings.c
> +++ b/fsmonitor-settings.c
> @@ -10,7 +10,9 @@
>   struct fsmonitor_settings {
>   	enum fsmonitor_mode mode;
>   	enum fsmonitor_reason reason;
> +	int allow_remote;
>   	char *hook_path;
> +	char *sock_dir;
>   };
>   
>   static enum fsmonitor_reason check_for_incompatible(struct repository *r)
> @@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
>   	CALLOC_ARRAY(s, 1);
>   	s->mode = FSMONITOR_MODE_DISABLED;
>   	s->reason = FSMONITOR_REASON_UNTESTED;
> +	s->allow_remote = -1;
>   
>   	return s;
>   }
> @@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)
>   		fsm_settings__set_disabled(r);
>   }
>   
> +int fsm_settings__get_allow_remote(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->allow_remote;
> +}
> +
> +const char *fsm_settings__get_socket_dir(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->sock_dir;
> +}
> +
>   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>   {
>   	if (!r)
> @@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>   	return r->settings.fsmonitor->mode;
>   }
>   
> +
>   const char *fsm_settings__get_hook_path(struct repository *r)
>   {
>   	if (!r)
> @@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
>   	return r->settings.fsmonitor->hook_path;
>   }
>   
> +void fsm_settings__set_allow_remote(struct repository *r)
> +{
> +	int allow;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
> +		r->settings.fsmonitor->allow_remote = allow;
> +
> +	return;
> +}
> +
> +void fsm_settings__set_socket_dir(struct repository *r)
> +{
> +	const char *path;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +
> +	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
> +		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
> +		r->settings.fsmonitor->sock_dir = strdup(path);
> +	}
> +
> +	return;
> +}
> +
>   void fsm_settings__set_ipc(struct repository *r)
>   {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);
>   
>   	if (reason != FSMONITOR_REASON_OK) {
>   		fsm_settings__set_incompatible(r, reason);
> @@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
>   
>   void fsm_settings__set_hook(struct repository *r, const char *path)
>   {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);

When we use the hook interface (to talk to a third-party
app like Watchman) we do not use the unix domain socket.
(We talk to the hook child using a normal pipe.)

FWIW, I added code in my series to limit the use of hook-based
fsmonitors to local filesystems for the paranoia-based
reasons I've described before.  It wasn't because of the UDS.

So we don't need to set the socket directory in this case.

>   
>   	if (reason != FSMONITOR_REASON_OK) {
>   		fsm_settings__set_incompatible(r, reason);
> diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
> index d9c2605197f..2de54c85e94 100644
> --- a/fsmonitor-settings.h
> +++ b/fsmonitor-settings.h
> @@ -23,12 +23,16 @@ enum fsmonitor_reason {
>   	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
>   };
>   
> +void fsm_settings__set_allow_remote(struct repository *r);
> +void fsm_settings__set_socket_dir(struct repository *r);
>   void fsm_settings__set_ipc(struct repository *r);
>   void fsm_settings__set_hook(struct repository *r, const char *path);
>   void fsm_settings__set_disabled(struct repository *r);
>   void fsm_settings__set_incompatible(struct repository *r,
>   				    enum fsmonitor_reason reason);
>   
> +int fsm_settings__get_allow_remote(struct repository *r);
> +const char *fsm_settings__get_socket_dir(struct repository *r);
>   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
>   const char *fsm_settings__get_hook_path(struct repository *r);
>   
> 

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
                           ` (2 preceding siblings ...)
  2022-09-01 17:53         ` Jeff Hostetler
@ 2022-09-01 18:04         ` Jeff Hostetler
  2022-09-01 21:21         ` Jeff Hostetler
  4 siblings, 0 replies; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-01 18:04 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git; +Cc: Eric DeCosta



On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>
> 
> Introduce two new configuration options
> 
>     fsmonitor.allowRemote - setting this to true overrides fsmonitor's
>     default behavior of erroring out when enountering network file
>     systems. Additionly, when true, the Unix domain socket (UDS) file
>     used for IPC is located in $HOME rather than in the .git directory.
> 
>     fsmonitor.socketDir - allows for the UDS file to be located
>     anywhere the user chooses rather $HOME.

If we add these config settings, we should update the docs
to describe them.

Jeff

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-08-31 16:09       ` [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path Eric DeCosta via GitGitGadget
  2022-08-31 19:37         ` Ævar Arnfjörð Bjarmason
@ 2022-09-01 20:05         ` Jeff Hostetler
  2022-09-02 16:35           ` Eric DeCosta
  1 sibling, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-01 20:05 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git; +Cc: Eric DeCosta



On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>
> 
> Consider the following network working directory that is mounted under
> /System/Volumes/Data:
> 
> /network/working/directory
> 
> The git working directory path is:
> 
> /System/Volumes/Data/network/working/directory
> 
> The paths reported by FSEvents always start with /network. fsmonitor
> expects paths to be under the working directory; therefore it
> fails to match /network/... and ignores the change.

I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted
directories with a path relative to the mount-point
rather than an absolute path?

In your example, is "/network/working/directory" a
valid path on your system (that also happens to be the
same directory as "/System/Volumes/...")

That is, do you have some aliasing going on where both paths
are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?


Did you have to "cd /System/Volumes/..." to get Git to have
the absolute path be this?  Or were you doing a "cd /network/..."?
(Sorry to ask so many questions but I don't have a pair of
systems to test any of this on right now.)

> 
> Change things such that if fsmonitor.allowRemote is true that the
> paths reported via FSEevents are normalized to the real path.
> 
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
>   compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++---
>   1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
> index 8e208e8289e..2ed828649ff 100644
> --- a/compat/fsmonitor/fsm-listen-darwin.c
> +++ b/compat/fsmonitor/fsm-listen-darwin.c
> @@ -26,6 +26,7 @@
>   #include "fsmonitor.h"
>   #include "fsm-listen.h"
>   #include "fsmonitor--daemon.h"
> +#include "fsmonitor-settings.h"
>   
>   struct fsm_listen_data
>   {
> @@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)
>   	free(composed);
>   }
>   
> -
>   static void fsevent_callback(ConstFSEventStreamRef streamRef,
>   			     void *ctx,
>   			     size_t num_of_events,
> @@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
>   		/*
>   		 * On Mac, we receive an array of absolute paths.
>   		 */
> -		path_k = paths[k];
> +		if (fsm_settings__get_allow_remote(the_repository) > 0) {
> +			strbuf_reset(&tmp);
> +			strbuf_realpath_forgiving(&tmp, paths[k], 0);
> +			path_k = tmp.buf;
> +		} else
> +			path_k = paths[k];

This feels wrong.

It is not that fsmonitor.allowRemote is true, but whether or not
this particular file system *IS* remote, right?

>   
>   		/*
>   		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
> @@ -237,6 +242,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
>   
>   			fsmonitor_force_resync(state);
>   			fsmonitor_batch__free_list(batch);
> +			batch = NULL;
>   			string_list_clear(&cookie_list, 0);
>   
>   			/*
> @@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
>   
>   		case IS_WORKDIR_PATH:
>   			/* try to queue normal pathnames */
> -
>   			if (trace_pass_fl(&trace_fsmonitor))
>   				log_flags_set(path_k, event_flags[k]);
>   
> 

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-08-31 16:09       ` [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir Eric DeCosta via GitGitGadget
                           ` (3 preceding siblings ...)
  2022-09-01 18:04         ` Jeff Hostetler
@ 2022-09-01 21:21         ` Jeff Hostetler
  2022-09-02 16:54           ` Eric DeCosta
  4 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-01 21:21 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git; +Cc: Eric DeCosta



On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> From: Eric DeCosta <edecosta@mathworks.com>
> 
> Introduce two new configuration options
> 
>     fsmonitor.allowRemote - setting this to true overrides fsmonitor's
>     default behavior of erroring out when enountering network file
>     systems. Additionly, when true, the Unix domain socket (UDS) file
>     used for IPC is located in $HOME rather than in the .git directory.
> 
>     fsmonitor.socketDir - allows for the UDS file to be located
>     anywhere the user chooses rather $HOME.
> 
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>

I hate to say it, but I think I'd like to start over on this
patch.  I think it adds too much to the upper layers.

Could we:

1. Move the GIT_PATH_FUNC() down into compat/fsmonitor/fsm-settings-*.c
    so that we don't need the ifdef-ing because we only really need to
    change the path on MacOS.

    Change it from this macro trick to be an actual function
    that computes the path and sticks it in a static buffer
    and returns it on future calls.  (We call it from several
    different places.)

    On MacOS have it always compute a $HOME or fsmonitor.socketDir
    based path.

    (This is called early, so we don't know anything about the
     r->settings.fsmonitor data yet (like whether we'll have an
     ipc or hook), so we don't know whether to worry about whether
     the socket-directory is local/remote yet.)

2. Get rid of the fsm_settings__get_allow_remote() and
    __get_socket_dir() functions in fsmonitor-settings.c.

    Besides, having it at this layer says that "allow-remote"
    is an all-or-nothing flag (that the platform-independent
    layer doesn't know anything about).  We may eventually find
    that we want the platform code to be smarter about that,
    such as allow remote from NFSv4 but not NFSv3.  We don't
    want to litter the platform-independent code with that.

    And it says that we always use a UDS.  We might change
    that later on MacOS.  And we don't need them at all on Windows.

3. Get rid of the fsm_settings__set_allow_remote() and
    __set_socket_dir().  The FSMonitor code only needs to access
    them once down in the fsm_os__incompatible() code, so we
    could just inline the relevant parts down there.

4. Leave the 'reason = check_for_incompatible()' as it was
    in fsm_setttings__set_ipc() and __set_hook().

    But you might pass a boolean "is-ipc" or "is-hook" to
    check_for_incompatible() that it could pass down to
    fsm_os__incompatible().  That might save the need to the
    fsmonitor.socketDir stuff when they want to use Watchman.

5. In fsm-settings-darwin.c:fsm_os__incompatible()
    complain if the socket path is remote (when is-ipc is set).

    This probably ought to be a new _REASON_ type for non-local
    UDS.  And this is independent of whether the actual worktree
    is remote or whether remote worktrees are allowed.

6. In fsm-settings-darwin.c:check_volume()

All that should be necessary is to:

     if (!(fs.f_flags & MNT_LOCAL))
+   {
+       // ... repo_config_get_bool(... "fsmonitor.allowremote" ...)
         return FSMONITOR_REASON_REMOTE;
+   }


Alternatively, for 5 & 6, we could pass a pathname to check_volume()
so that fsm_os__incompatible() calls it twice:  once for the worktree
and once for the socketdir.

The ntfs and msdos restrictions were for UDS reasons, so you might
want pass some flags to check_volume() too.

Sorry for the redesign and I hope this all makes sense,
Jeff


> ---
>   fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++--
>   fsmonitor-settings.h |  4 +++
>   2 files changed, 69 insertions(+), 2 deletions(-)
> 
> diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
> index 464424a1e92..a15eeecebf4 100644
> --- a/fsmonitor-settings.c
> +++ b/fsmonitor-settings.c
> @@ -10,7 +10,9 @@
>   struct fsmonitor_settings {
>   	enum fsmonitor_mode mode;
>   	enum fsmonitor_reason reason;
> +	int allow_remote;
>   	char *hook_path;
> +	char *sock_dir;
>   };
>   
>   static enum fsmonitor_reason check_for_incompatible(struct repository *r)
> @@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)
>   	CALLOC_ARRAY(s, 1);
>   	s->mode = FSMONITOR_MODE_DISABLED;
>   	s->reason = FSMONITOR_REASON_UNTESTED;
> +	s->allow_remote = -1;
>   
>   	return s;
>   }
> @@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)
>   		fsm_settings__set_disabled(r);
>   }
>   
> +int fsm_settings__get_allow_remote(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->allow_remote;
> +}
> +
> +const char *fsm_settings__get_socket_dir(struct repository *r)
> +{
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		lookup_fsmonitor_settings(r);
> +
> +	return r->settings.fsmonitor->sock_dir;
> +}
> +
>   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>   {
>   	if (!r)
> @@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>   	return r->settings.fsmonitor->mode;
>   }
>   
> +
>   const char *fsm_settings__get_hook_path(struct repository *r)
>   {
>   	if (!r)
> @@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)
>   	return r->settings.fsmonitor->hook_path;
>   }
>   
> +void fsm_settings__set_allow_remote(struct repository *r)
> +{
> +	int allow;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
> +		r->settings.fsmonitor->allow_remote = allow;
> +
> +	return;
> +}
> +
> +void fsm_settings__set_socket_dir(struct repository *r)
> +{
> +	const char *path;
> +
> +	if (!r)
> +		r = the_repository;
> +	if (!r->settings.fsmonitor)
> +		r->settings.fsmonitor = alloc_settings();
> +
> +	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
> +		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
> +		r->settings.fsmonitor->sock_dir = strdup(path);
> +	}
> +
> +	return;
> +}
> +
>   void fsm_settings__set_ipc(struct repository *r)
>   {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);
>   
>   	if (reason != FSMONITOR_REASON_OK) {
>   		fsm_settings__set_incompatible(r, reason);
> @@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)
>   
>   void fsm_settings__set_hook(struct repository *r, const char *path)
>   {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason;
> +
> +	fsm_settings__set_allow_remote(r);
> +	fsm_settings__set_socket_dir(r);
> +	reason = check_for_incompatible(r);
>   
>   	if (reason != FSMONITOR_REASON_OK) {
>   		fsm_settings__set_incompatible(r, reason);
> diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
> index d9c2605197f..2de54c85e94 100644
> --- a/fsmonitor-settings.h
> +++ b/fsmonitor-settings.h
> @@ -23,12 +23,16 @@ enum fsmonitor_reason {
>   	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
>   };
>   
> +void fsm_settings__set_allow_remote(struct repository *r);
> +void fsm_settings__set_socket_dir(struct repository *r);
>   void fsm_settings__set_ipc(struct repository *r);
>   void fsm_settings__set_hook(struct repository *r, const char *path);
>   void fsm_settings__set_disabled(struct repository *r);
>   void fsm_settings__set_incompatible(struct repository *r,
>   				    enum fsmonitor_reason reason);
>   
> +int fsm_settings__get_allow_remote(struct repository *r);
> +const char *fsm_settings__get_socket_dir(struct repository *r);
>   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
>   const char *fsm_settings__get_hook_path(struct repository *r);
>   
> 

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-01 20:05         ` Jeff Hostetler
@ 2022-09-02 16:35           ` Eric DeCosta
  2022-09-06 17:13             ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta @ 2022-09-02 16:35 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Thursday, September 1, 2022 4:06 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org
> Cc: Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the
> real path
> 
> 
> 
> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> > From: Eric DeCosta <edecosta@mathworks.com>
> >
> > Consider the following network working directory that is mounted under
> > /System/Volumes/Data:
> >
> > /network/working/directory
> >
> > The git working directory path is:
> >
> > /System/Volumes/Data/network/working/directory
> >
> > The paths reported by FSEvents always start with /network. fsmonitor
> > expects paths to be under the working directory; therefore it fails to
> > match /network/... and ignores the change.
> 
> I'm not sure I understand what's going on here.
> Are you saying that FSEvents reports network mounted directories with a
> path relative to the mount-point rather than an absolute path?
>

Yes

> In your example, is "/network/working/directory" a valid path on your
> system (that also happens to be the same directory as
> "/System/Volumes/...")
> 
> That is, do you have some aliasing going on where both paths are valid -- like
> a pair of hard-linked directories?
> Or, is there something special about a network mount point?
> 
> 
> Did you have to "cd /System/Volumes/..." to get Git to have the absolute
> path be this?  Or were you doing a "cd /network/..."?
> (Sorry to ask so many questions but I don't have a pair of systems to test any
> of this on right now.)
> 

 "/network/working/directory" is mounted under 
"/System/Volumes/Data/network/working/directory"

There is also a symlink:

"/network" -> "/System/Volumes/Data/network"

No matter if I "cd /System/Volumes/Data/network/working/directory"
or "cd /network/working/directory" the paths reported by FSEvents
always start with "/network/working/directory"

If I do a similar symlink with local directories, I do not get this
unexpected behavior.

I need to remove the symlink and try it that way, but I need to
coordinate with the machine's owner.

> >
> > Change things such that if fsmonitor.allowRemote is true that the
> > paths reported via FSEevents are normalized to the real path.
> >
> > Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> > ---
> >   compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++---
> >   1 file changed, 8 insertions(+), 3 deletions(-)
> >
> > diff --git a/compat/fsmonitor/fsm-listen-darwin.c
> > b/compat/fsmonitor/fsm-listen-darwin.c
> > index 8e208e8289e..2ed828649ff 100644
> > --- a/compat/fsmonitor/fsm-listen-darwin.c
> > +++ b/compat/fsmonitor/fsm-listen-darwin.c
> > @@ -26,6 +26,7 @@
> >   #include "fsmonitor.h"
> >   #include "fsm-listen.h"
> >   #include "fsmonitor--daemon.h"
> > +#include "fsmonitor-settings.h"
> >
> >   struct fsm_listen_data
> >   {
> > @@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch
> *batch, const char *path)
> >   	free(composed);
> >   }
> >
> > -
> >   static void fsevent_callback(ConstFSEventStreamRef streamRef,
> >   			     void *ctx,
> >   			     size_t num_of_events,
> > @@ -209,7 +209,12 @@ static void
> fsevent_callback(ConstFSEventStreamRef streamRef,
> >   		/*
> >   		 * On Mac, we receive an array of absolute paths.
> >   		 */
> > -		path_k = paths[k];
> > +		if (fsm_settings__get_allow_remote(the_repository) > 0) {
> > +			strbuf_reset(&tmp);
> > +			strbuf_realpath_forgiving(&tmp, paths[k], 0);
> > +			path_k = tmp.buf;
> > +		} else
> > +			path_k = paths[k];
> 
> This feels wrong.
> 
> It is not that fsmonitor.allowRemote is true, but whether or not this
> particular file system *IS* remote, right?

True. I suppose each path could be checked, or some bit could be set
if the working directory is remote, perhaps along the lines of
fsmonitor_ipc__get_path. Then the determination of the IPC path
and whether it is remote would be in one place. fsm-settings-*
would then just get that bit and check it against allowRemote.

Thoughts?


> >
> >   		/*
> >   		 * If you want to debug FSEvents, log them to
> GIT_TRACE_FSMONITOR.
> > @@ -237,6 +242,7 @@ static void
> fsevent_callback(ConstFSEventStreamRef
> > streamRef,
> >
> >   			fsmonitor_force_resync(state);
> >   			fsmonitor_batch__free_list(batch);
> > +			batch = NULL;
> >   			string_list_clear(&cookie_list, 0);
> >
> >   			/*
> > @@ -313,7 +319,6 @@ static void
> fsevent_callback(ConstFSEventStreamRef
> > streamRef,
> >
> >   		case IS_WORKDIR_PATH:
> >   			/* try to queue normal pathnames */
> > -
> >   			if (trace_pass_fl(&trace_fsmonitor))
> >   				log_flags_set(path_k, event_flags[k]);
> >
> >


^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-09-01 21:21         ` Jeff Hostetler
@ 2022-09-02 16:54           ` Eric DeCosta
  2022-09-06 14:27             ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta @ 2022-09-02 16:54 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Thursday, September 1, 2022 5:22 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org
> Cc: Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v4 1/4] fsmonitor: add two new config options,
> allowRemote and socketDir
> 
> 
> 
> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> > From: Eric DeCosta <edecosta@mathworks.com>
> >
> > Introduce two new configuration options
> >
> >     fsmonitor.allowRemote - setting this to true overrides fsmonitor's
> >     default behavior of erroring out when enountering network file
> >     systems. Additionly, when true, the Unix domain socket (UDS) file
> >     used for IPC is located in $HOME rather than in the .git directory.
> >
> >     fsmonitor.socketDir - allows for the UDS file to be located
> >     anywhere the user chooses rather $HOME.
> >
> > Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> 
> I hate to say it, but I think I'd like to start over on this patch.  I think it adds
> too much to the upper layers.
> 
> Could we:
> 
> 1. Move the GIT_PATH_FUNC() down into compat/fsmonitor/fsm-settings-*.c
>     so that we don't need the ifdef-ing because we only really need to
>     change the path on MacOS.
> 
>     Change it from this macro trick to be an actual function
>     that computes the path and sticks it in a static buffer
>     and returns it on future calls.  (We call it from several
>     different places.)
> 
>     On MacOS have it always compute a $HOME or fsmonitor.socketDir
>     based path.
> 
>     (This is called early, so we don't know anything about the
>      r->settings.fsmonitor data yet (like whether we'll have an
>      ipc or hook), so we don't know whether to worry about whether
>      the socket-directory is local/remote yet.)
> 
> 2. Get rid of the fsm_settings__get_allow_remote() and
>     __get_socket_dir() functions in fsmonitor-settings.c.
> 
>     Besides, having it at this layer says that "allow-remote"
>     is an all-or-nothing flag (that the platform-independent
>     layer doesn't know anything about).  We may eventually find
>     that we want the platform code to be smarter about that,
>     such as allow remote from NFSv4 but not NFSv3.  We don't
>     want to litter the platform-independent code with that.
> 
>     And it says that we always use a UDS.  We might change
>     that later on MacOS.  And we don't need them at all on Windows.
> 
> 3. Get rid of the fsm_settings__set_allow_remote() and
>     __set_socket_dir().  The FSMonitor code only needs to access
>     them once down in the fsm_os__incompatible() code, so we
>     could just inline the relevant parts down there.
 
Got it.

> 4. Leave the 'reason = check_for_incompatible()' as it was
>     in fsm_setttings__set_ipc() and __set_hook().
> 
>     But you might pass a boolean "is-ipc" or "is-hook" to
>     check_for_incompatible() that it could pass down to
>     fsm_os__incompatible().  That might save the need to the
>     fsmonitor.socketDir stuff when they want to use Watchman.

I see, avoid the UDS check altogether if "is-hook"

> 5. In fsm-settings-darwin.c:fsm_os__incompatible()
>     complain if the socket path is remote (when is-ipc is set).
> 
>     This probably ought to be a new _REASON_ type for non-local
>     UDS.  And this is independent of whether the actual worktree
>     is remote or whether remote worktrees are allowed.

I thought that was what FSMONITOR_REASON_NOSOCKETS was for?

> 6. In fsm-settings-darwin.c:check_volume()
> 
> All that should be necessary is to:
> 
>      if (!(fs.f_flags & MNT_LOCAL))
> +   {
> +       // ... repo_config_get_bool(... "fsmonitor.allowremote" ...)
>          return FSMONITOR_REASON_REMOTE;
> +   }
> 
> 
> Alternatively, for 5 & 6, we could pass a pathname to check_volume() so that
> fsm_os__incompatible() calls it twice:  once for the worktree and once for
> the socketdir.
> 
> The ntfs and msdos restrictions were for UDS reasons, so you might want
> pass some flags to check_volume() too.
> 
> Sorry for the redesign and I hope this all makes sense, Jeff
> 

I've taken a first pass at all of this, will work on it some more.

-Eric

> 
> > ---
> >   fsmonitor-settings.c | 67
> ++++++++++++++++++++++++++++++++++++++++++--
> >   fsmonitor-settings.h |  4 +++
> >   2 files changed, 69 insertions(+), 2 deletions(-)
> >
> > diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c index
> > 464424a1e92..a15eeecebf4 100644
> > --- a/fsmonitor-settings.c
> > +++ b/fsmonitor-settings.c
> > @@ -10,7 +10,9 @@
> >   struct fsmonitor_settings {
> >   	enum fsmonitor_mode mode;
> >   	enum fsmonitor_reason reason;
> > +	int allow_remote;
> >   	char *hook_path;
> > +	char *sock_dir;
> >   };
> >
> >   static enum fsmonitor_reason check_for_incompatible(struct
> > repository *r) @@ -43,6 +45,7 @@ static struct fsmonitor_settings
> *alloc_settings(void)
> >   	CALLOC_ARRAY(s, 1);
> >   	s->mode = FSMONITOR_MODE_DISABLED;
> >   	s->reason = FSMONITOR_REASON_UNTESTED;
> > +	s->allow_remote = -1;
> >
> >   	return s;
> >   }
> > @@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct
> repository *r)
> >   		fsm_settings__set_disabled(r);
> >   }
> >
> > +int fsm_settings__get_allow_remote(struct repository *r) {
> > +	if (!r)
> > +		r = the_repository;
> > +	if (!r->settings.fsmonitor)
> > +		lookup_fsmonitor_settings(r);
> > +
> > +	return r->settings.fsmonitor->allow_remote;
> > +}
> > +
> > +const char *fsm_settings__get_socket_dir(struct repository *r) {
> > +	if (!r)
> > +		r = the_repository;
> > +	if (!r->settings.fsmonitor)
> > +		lookup_fsmonitor_settings(r);
> > +
> > +	return r->settings.fsmonitor->sock_dir; }
> > +
> >   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
> >   {
> >   	if (!r)
> > @@ -100,6 +123,7 @@ enum fsmonitor_mode
> fsm_settings__get_mode(struct repository *r)
> >   	return r->settings.fsmonitor->mode;
> >   }
> >
> > +
> >   const char *fsm_settings__get_hook_path(struct repository *r)
> >   {
> >   	if (!r)
> > @@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct
> repository *r)
> >   	return r->settings.fsmonitor->hook_path;
> >   }
> >
> > +void fsm_settings__set_allow_remote(struct repository *r) {
> > +	int allow;
> > +
> > +	if (!r)
> > +		r = the_repository;
> > +	if (!r->settings.fsmonitor)
> > +		r->settings.fsmonitor = alloc_settings();
> > +	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
> > +		r->settings.fsmonitor->allow_remote = allow;
> > +
> > +	return;
> > +}
> > +
> > +void fsm_settings__set_socket_dir(struct repository *r) {
> > +	const char *path;
> > +
> > +	if (!r)
> > +		r = the_repository;
> > +	if (!r->settings.fsmonitor)
> > +		r->settings.fsmonitor = alloc_settings();
> > +
> > +	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
> > +		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
> > +		r->settings.fsmonitor->sock_dir = strdup(path);
> > +	}
> > +
> > +	return;
> > +}
> > +
> >   void fsm_settings__set_ipc(struct repository *r)
> >   {
> > -	enum fsmonitor_reason reason = check_for_incompatible(r);
> > +	enum fsmonitor_reason reason;
> > +
> > +	fsm_settings__set_allow_remote(r);
> > +	fsm_settings__set_socket_dir(r);
> > +	reason = check_for_incompatible(r);
> >
> >   	if (reason != FSMONITOR_REASON_OK) {
> >   		fsm_settings__set_incompatible(r, reason); @@ -135,7
> +194,11 @@
> > void fsm_settings__set_ipc(struct repository *r)
> >
> >   void fsm_settings__set_hook(struct repository *r, const char *path)
> >   {
> > -	enum fsmonitor_reason reason = check_for_incompatible(r);
> > +	enum fsmonitor_reason reason;
> > +
> > +	fsm_settings__set_allow_remote(r);
> > +	fsm_settings__set_socket_dir(r);
> > +	reason = check_for_incompatible(r);
> >
> >   	if (reason != FSMONITOR_REASON_OK) {
> >   		fsm_settings__set_incompatible(r, reason); diff --git
> > a/fsmonitor-settings.h b/fsmonitor-settings.h index
> > d9c2605197f..2de54c85e94 100644
> > --- a/fsmonitor-settings.h
> > +++ b/fsmonitor-settings.h
> > @@ -23,12 +23,16 @@ enum fsmonitor_reason {
> >   	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support
> Unix sockets */
> >   };
> >
> > +void fsm_settings__set_allow_remote(struct repository *r); void
> > +fsm_settings__set_socket_dir(struct repository *r);
> >   void fsm_settings__set_ipc(struct repository *r);
> >   void fsm_settings__set_hook(struct repository *r, const char *path);
> >   void fsm_settings__set_disabled(struct repository *r);
> >   void fsm_settings__set_incompatible(struct repository *r,
> >   				    enum fsmonitor_reason reason);
> >
> > +int fsm_settings__get_allow_remote(struct repository *r); const char
> > +*fsm_settings__get_socket_dir(struct repository *r);
> >   enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
> >   const char *fsm_settings__get_hook_path(struct repository *r);
> >
> >


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
  2022-09-02 16:54           ` Eric DeCosta
@ 2022-09-06 14:27             ` Jeff Hostetler
  0 siblings, 0 replies; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-06 14:27 UTC (permalink / raw)
  To: Eric DeCosta, Eric DeCosta via GitGitGadget, git



On 9/2/22 12:54 PM, Eric DeCosta wrote:
> 
> 
>> -----Original Message-----
>> From: Jeff Hostetler <git@jeffhostetler.com>
>> Sent: Thursday, September 1, 2022 5:22 PM
>> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
>> git@vger.kernel.org
>> Cc: Eric DeCosta <edecosta@mathworks.com>
>> Subject: Re: [PATCH v4 1/4] fsmonitor: add two new config options,
>> allowRemote and socketDir
>>
>>
>>
>> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
>>> From: Eric DeCosta <edecosta@mathworks.com>
>>>
>>> Introduce two new configuration options
[...]
>> 5. In fsm-settings-darwin.c:fsm_os__incompatible()
>>      complain if the socket path is remote (when is-ipc is set).
>>
>>      This probably ought to be a new _REASON_ type for non-local
>>      UDS.  And this is independent of whether the actual worktree
>>      is remote or whether remote worktrees are allowed.
> 
> I thought that was what FSMONITOR_REASON_NOSOCKETS was for?

Yeah, I was probably juggling too many things in my
head when I was making that list.  And I'm sure it'll
fall our when you see what the code looks like.  The
_NOSOCKETS case should basically be just the places where
you test the path of the socket-dir (if it is always outside
of the .git dir).  None of the tests on the actual workdir
inspection should hit it anymore, right?

And we can save for a later day whether or not to test
for NTFS and FAT32 (local or remote) in the Darwin code
based on whether or not the kernel sends FSEvents for
them.

Jeff


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-02 16:35           ` Eric DeCosta
@ 2022-09-06 17:13             ` Jeff Hostetler
  2022-09-06 19:02               ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-06 17:13 UTC (permalink / raw)
  To: Eric DeCosta, Eric DeCosta via GitGitGadget, git



On 9/2/22 12:35 PM, Eric DeCosta wrote:
> 
> 
>> -----Original Message-----
>> From: Jeff Hostetler <git@jeffhostetler.com>
>> Sent: Thursday, September 1, 2022 4:06 PM
>> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
>> git@vger.kernel.org
>> Cc: Eric DeCosta <edecosta@mathworks.com>
>> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the
>> real path
>>
>>
>>
>> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
>>> From: Eric DeCosta <edecosta@mathworks.com>
>>>
>>> Consider the following network working directory that is mounted under
>>> /System/Volumes/Data:
>>>
>>> /network/working/directory
>>>
>>> The git working directory path is:
>>>
>>> /System/Volumes/Data/network/working/directory
>>>
>>> The paths reported by FSEvents always start with /network. fsmonitor
>>> expects paths to be under the working directory; therefore it fails to
>>> match /network/... and ignores the change.
>>
>> I'm not sure I understand what's going on here.
>> Are you saying that FSEvents reports network mounted directories with a
>> path relative to the mount-point rather than an absolute path?
>>
> 
> Yes
> 
>> In your example, is "/network/working/directory" a valid path on your
>> system (that also happens to be the same directory as
>> "/System/Volumes/...")
>>
>> That is, do you have some aliasing going on where both paths are valid -- like
>> a pair of hard-linked directories?
>> Or, is there something special about a network mount point?
>>
>>
>> Did you have to "cd /System/Volumes/..." to get Git to have the absolute
>> path be this?  Or were you doing a "cd /network/..."?
>> (Sorry to ask so many questions but I don't have a pair of systems to test any
>> of this on right now.)
>>
> 
>   "/network/working/directory" is mounted under
> "/System/Volumes/Data/network/working/directory"
> 
> There is also a symlink:
> 
> "/network" -> "/System/Volumes/Data/network"
> 
> No matter if I "cd /System/Volumes/Data/network/working/directory"
> or "cd /network/working/directory" the paths reported by FSEvents
> always start with "/network/working/directory"
> 
> If I do a similar symlink with local directories, I do not get this
> unexpected behavior.
> 
> I need to remove the symlink and try it that way, but I need to
> coordinate with the machine's owner.


I think you have stumbled upon a recent MacOS feature called
"firmlinks".  I'm just reading up on it myself, so I shouldn't
speculate here (yet), but maybe you too could do some reading
on the subject.

This makes me wonder if the symlink is historical.  The real
magic is in the firmlinks.  For example, if I do:

	$ (cd / ; ls -l | grep Users)
	drwxr-xr-x   6 root  admin   192 Apr  6  2020 Users

	$ (cd /Users ; df .)
	Filesystem   512-blocks      Used Available Capacity iused      ifree 
%iused  Mounted on
	/dev/disk1s1  976490576 608954344 338850488    65% 4181323 4878271557 
  0%   /System/Volumes/Data

we can see that /Users is on /System/Volumes/Data (and there is a
/System/Volumes/Data/Users directory with the same metadata), but
it is not a symlink.


See [1] for more info.

[1] 
http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-magic.html


[...]
>>> @@ -209,7 +209,12 @@ static void
>> fsevent_callback(ConstFSEventStreamRef streamRef,
>>>    		/*
>>>    		 * On Mac, we receive an array of absolute paths.
>>>    		 */
>>> -		path_k = paths[k];
>>> +		if (fsm_settings__get_allow_remote(the_repository) > 0) {
>>> +			strbuf_reset(&tmp);
>>> +			strbuf_realpath_forgiving(&tmp, paths[k], 0);
>>> +			path_k = tmp.buf;
>>> +		} else
>>> +			path_k = paths[k];
>>
>> This feels wrong.
>>
>> It is not that fsmonitor.allowRemote is true, but whether or not this
>> particular file system *IS* remote, right?
> 
> True. I suppose each path could be checked, or some bit could be set
> if the working directory is remote, perhaps along the lines of
> fsmonitor_ipc__get_path. Then the determination of the IPC path
> and whether it is remote would be in one place. fsm-settings-*
> would then just get that bit and check it against allowRemote.
> 
> Thoughts?

I'll do some digging here.  There ought to be a way to tell if a
component directory in a pathname has a firmlink peer.  (But I'm
traveling for GitMerge, so I might not have time to look at this
until afterwards.)

This would indicate that a "bi-directional wormhole" (their words)
alias is available (and hopefully, a way to computer the other peer....)

I'm thinking that the "/network/..." path in the FSEvents is because
FSEventD is using a particular peer spelling (that we weren't
expecting).

If we can compute the spellings of the peers once when the daemon
starts up (and maybe make a little dictionary), then we can do
prefix tricks on the absolute path after the
         path_k = paths[k];
step in fsevent_callback() to extract a relative path rather than
an absolute path.

Then call fsmonitor_classify_path_relative() instead of _absolute().

Watch out though, there are several places that do
         rel = path+k + ...state->path_worktree_watch.len...
that would need to be adjusted.

Hope this helps,
Jeff






^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-06 17:13             ` Jeff Hostetler
@ 2022-09-06 19:02               ` Eric DeCosta
  2022-09-06 19:33                 ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta @ 2022-09-06 19:02 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Tuesday, September 6, 2022 1:14 PM
> To: Eric DeCosta <edecosta@mathworks.com>; Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com>; git@vger.kernel.org
> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to
> the real path
> 
> 
> 
> On 9/2/22 12:35 PM, Eric DeCosta wrote:
> >
> >
> >> -----Original Message-----
> >> From: Jeff Hostetler <git@jeffhostetler.com>
> >> Sent: Thursday, September 1, 2022 4:06 PM
> >> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> >> git@vger.kernel.org
> >> Cc: Eric DeCosta <edecosta@mathworks.com>
> >> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths
> >> to the real path
> >>
> >>
> >>
> >> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> >>> From: Eric DeCosta <edecosta@mathworks.com>
> >>>
> >>> Consider the following network working directory that is mounted
> >>> under
> >>> /System/Volumes/Data:
> >>>
> >>> /network/working/directory
> >>>
> >>> The git working directory path is:
> >>>
> >>> /System/Volumes/Data/network/working/directory
> >>>
> >>> The paths reported by FSEvents always start with /network. fsmonitor
> >>> expects paths to be under the working directory; therefore it fails
> >>> to match /network/... and ignores the change.
> >>
> >> I'm not sure I understand what's going on here.
> >> Are you saying that FSEvents reports network mounted directories with
> >> a path relative to the mount-point rather than an absolute path?
> >>
> >
> > Yes
> >
> >> In your example, is "/network/working/directory" a valid path on your
> >> system (that also happens to be the same directory as
> >> "/System/Volumes/...")
> >>
> >> That is, do you have some aliasing going on where both paths are
> >> valid -- like a pair of hard-linked directories?
> >> Or, is there something special about a network mount point?
> >>
> >>
> >> Did you have to "cd /System/Volumes/..." to get Git to have the
> >> absolute path be this? Or were you doing a "cd /network/..."?
> >> (Sorry to ask so many questions but I don't have a pair of systems to
> >> test any of this on right now.)
> >>
> >
> > "/network/working/directory" is mounted under
> > "/System/Volumes/Data/network/working/directory"
> >
> > There is also a symlink:
> >
> > "/network" -> "/System/Volumes/Data/network"
> >
> > No matter if I "cd /System/Volumes/Data/network/working/directory"
> > or "cd /network/working/directory" the paths reported by FSEvents
> > always start with "/network/working/directory"
> >
> > If I do a similar symlink with local directories, I do not get this
> > unexpected behavior.
> >
> > I need to remove the symlink and try it that way, but I need to
> > coordinate with the machine's owner.
> 
> 
> I think you have stumbled upon a recent MacOS feature called "firmlinks".
> I'm just reading up on it myself, so I shouldn't speculate here (yet), but
> maybe you too could do some reading on the subject.
> 
> This makes me wonder if the symlink is historical. The real magic is in the
> firmlinks. For example, if I do:
> 
> $ (cd / ; ls -l | grep Users)
> drwxr-xr-x 6 root admin 192 Apr 6 2020 Users
> 
> $ (cd /Users ; df .)
> Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted
> on
> /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 0%
> /System/Volumes/Data
> 
> we can see that /Users is on /System/Volumes/Data (and there is a
> /System/Volumes/Data/Users directory with the same metadata), but it is
> not a symlink.
> 
> 
> See [1] for more info.
> 
> [1]
> http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
> magic.html <https://protect-
> us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co
> m>
> 
> 

Hmm, I don't see anything that suggests a firmlink is involved but this is new to me so maybe I just don't see it. There is an automount in addition to the symlink though.

So, dispensing with the abstract "/network" path, and getting to the details of my environment:

% mount | grep /mathworks
map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks (autofs, automounted, nobrowse)
triggered map auto.sfs.sol2.devel on /System/Volumes/Data/mathworks/devel (autofs, automounted, nobrowse)
triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted, nobrowse)
triggered map sfs.worldwide.US-Natick.devel.sbs.29 on /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted, nobrowse)
batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2 (nfs, nodev, automounted, noatime, nobrowse)
...

% ls -l / | grep mathworks
lrwxr-xr-x   1 root  wheel    35 Aug 29 10:25 home@ -> /System/Volumes/Data/mathworks/home
lrwxr-xr-x   1 root  wheel    30 Aug 29 10:25 mathworks@ -> /System/Volumes/Data/mathworks

My worktree is: /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2
or, equivalently via the symlink just: /mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2

I have sudo'ers access now; so I can try mounting things manually, mess about with symlinks and see what I get.

> [...]
> >>> @@ -209,7 +209,12 @@ static void
> >> fsevent_callback(ConstFSEventStreamRef streamRef,
> >>> /*
> >>> * On Mac, we receive an array of absolute paths.
> >>> */
> >>> - path_k = paths[k];
> >>> + if (fsm_settings__get_allow_remote(the_repository) > 0) {
> >>> + strbuf_reset(&tmp);
> >>> + strbuf_realpath_forgiving(&tmp, paths[k], 0);
> >>> + path_k = tmp.buf;
> >>> + } else
> >>> + path_k = paths[k];
> >>
> >> This feels wrong.
> >>
> >> It is not that fsmonitor.allowRemote is true, but whether or not this
> >> particular file system *IS* remote, right?
> >
> > True. I suppose each path could be checked, or some bit could be set
> > if the working directory is remote, perhaps along the lines of
> > fsmonitor_ipc__get_path. Then the determination of the IPC path
> > and whether it is remote would be in one place. fsm-settings-*
> > would then just get that bit and check it against allowRemote.
> >
> > Thoughts?
> 
> I'll do some digging here. There ought to be a way to tell if a
> component directory in a pathname has a firmlink peer. (But I'm
> traveling for GitMerge, so I might not have time to look at this
> until afterwards.)
> 
> This would indicate that a "bi-directional wormhole" (their words)
> alias is available (and hopefully, a way to computer the other peer....)
> 
> I'm thinking that the "/network/..." path in the FSEvents is because
> FSEventD is using a particular peer spelling (that we weren't
> expecting).
> 
> If we can compute the spellings of the peers once when the daemon
> starts up (and maybe make a little dictionary), then we can do
> prefix tricks on the absolute path after the
> path_k = paths[k];
> step in fsevent_callback() to extract a relative path rather than
> an absolute path.
> 
> Then call fsmonitor_classify_path_relative() instead of _absolute().
> 
> Watch out though, there are several places that do
> rel = path+k + ...state->path_worktree_watch.len...
> that would need to be adjusted.
> 
> Hope this helps,
> Jeff
> 
> 
Thanks for the insights, I'll dig around and test things out some more.

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-06 19:02               ` Eric DeCosta
@ 2022-09-06 19:33                 ` Eric DeCosta
  2022-09-06 22:11                   ` Eric DeCosta
  2022-09-07 19:14                   ` Jeff Hostetler
  0 siblings, 2 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-06 19:33 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Eric DeCosta
> Sent: Tuesday, September 6, 2022 3:02 PM
> To: Jeff Hostetler <git@jeffhostetler.com>; Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com>; git@vger.kernel.org
> Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the
> real path
> 
> 
> 
> > -----Original Message-----
> > From: Jeff Hostetler <git@jeffhostetler.com>
> > Sent: Tuesday, September 6, 2022 1:14 PM
> > To: Eric DeCosta <edecosta@mathworks.com>; Eric DeCosta via
> > GitGitGadget <gitgitgadget@gmail.com>; git@vger.kernel.org
> > Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths
> > to the real path
> >
> >
> >
> > On 9/2/22 12:35 PM, Eric DeCosta wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Jeff Hostetler <git@jeffhostetler.com>
> > >> Sent: Thursday, September 1, 2022 4:06 PM
> > >> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> > >> git@vger.kernel.org
> > >> Cc: Eric DeCosta <edecosta@mathworks.com>
> > >> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event
> > >> paths to the real path
> > >>
> > >>
> > >>
> > >> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> > >>> From: Eric DeCosta <edecosta@mathworks.com>
> > >>>
> > >>> Consider the following network working directory that is mounted
> > >>> under
> > >>> /System/Volumes/Data:
> > >>>
> > >>> /network/working/directory
> > >>>
> > >>> The git working directory path is:
> > >>>
> > >>> /System/Volumes/Data/network/working/directory
> > >>>
> > >>> The paths reported by FSEvents always start with /network.
> > >>> fsmonitor expects paths to be under the working directory;
> > >>> therefore it fails to match /network/... and ignores the change.
> > >>
> > >> I'm not sure I understand what's going on here.
> > >> Are you saying that FSEvents reports network mounted directories
> > >> with a path relative to the mount-point rather than an absolute path?
> > >>
> > >
> > > Yes
> > >
> > >> In your example, is "/network/working/directory" a valid path on
> > >> your system (that also happens to be the same directory as
> > >> "/System/Volumes/...")
> > >>
> > >> That is, do you have some aliasing going on where both paths are
> > >> valid -- like a pair of hard-linked directories?
> > >> Or, is there something special about a network mount point?
> > >>
> > >>
> > >> Did you have to "cd /System/Volumes/..." to get Git to have the
> > >> absolute path be this? Or were you doing a "cd /network/..."?
> > >> (Sorry to ask so many questions but I don't have a pair of systems
> > >> to test any of this on right now.)
> > >>
> > >
> > > "/network/working/directory" is mounted under
> > > "/System/Volumes/Data/network/working/directory"
> > >
> > > There is also a symlink:
> > >
> > > "/network" -> "/System/Volumes/Data/network"
> > >
> > > No matter if I "cd /System/Volumes/Data/network/working/directory"
> > > or "cd /network/working/directory" the paths reported by FSEvents
> > > always start with "/network/working/directory"
> > >
> > > If I do a similar symlink with local directories, I do not get this
> > > unexpected behavior.
> > >
> > > I need to remove the symlink and try it that way, but I need to
> > > coordinate with the machine's owner.
> >
> >
> > I think you have stumbled upon a recent MacOS feature called "firmlinks".
> > I'm just reading up on it myself, so I shouldn't speculate here (yet),
> > but maybe you too could do some reading on the subject.
> >
> > This makes me wonder if the symlink is historical. The real magic is
> > in the firmlinks. For example, if I do:
> >
> > $ (cd / ; ls -l | grep Users)
> > drwxr-xr-x 6 root admin 192 Apr 6 2020 Users
> >
> > $ (cd /Users ; df .)
> > Filesystem 512-blocks Used Available Capacity iused ifree %iused
> > Mounted on
> > /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 0%
> > /System/Volumes/Data
> >
> > we can see that /Users is on /System/Volumes/Data (and there is a
> > /System/Volumes/Data/Users directory with the same metadata), but it
> > is not a symlink.
> >
> >
> > See [1] for more info.
> >
> > [1]
> > http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
> > magic.html <https://protect-
> >
> us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co
> > m>
> >
> >
> 
> Hmm, I don't see anything that suggests a firmlink is involved but this is new
> to me so maybe I just don't see it. There is an automount in addition to the
> symlink though.
> 
> So, dispensing with the abstract "/network" path, and getting to the details
> of my environment:
> 
> % mount | grep /mathworks
> map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks
> (autofs, automounted, nobrowse) triggered map auto.sfs.sol2.devel on
> /System/Volumes/Data/mathworks/devel (autofs, automounted,
> nobrowse) triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on
> /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted,
> nobrowse) triggered map sfs.worldwide.US-Natick.devel.sbs.29 on
> /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted,
> nobrowse)
> batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on
> /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
> 28560.2 (nfs, nodev, automounted, noatime, nobrowse) ...
> 
> % ls -l / | grep mathworks
> lrwxr-xr-x   1 root  wheel    35 Aug 29 10:25 home@ ->
> /System/Volumes/Data/mathworks/home
> lrwxr-xr-x   1 root  wheel    30 Aug 29 10:25 mathworks@ ->
> /System/Volumes/Data/mathworks
> 
> My worktree is:
> /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
> 28560.2
> or, equivalently via the symlink just:
> /mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2
> 
> I have sudo'ers access now; so I can try mounting things manually, mess
> about with symlinks and see what I get.
> 
> > [...]
> > >>> @@ -209,7 +209,12 @@ static void
> > >> fsevent_callback(ConstFSEventStreamRef streamRef,
> > >>> /*
> > >>> * On Mac, we receive an array of absolute paths.
> > >>> */
> > >>> - path_k = paths[k];
> > >>> + if (fsm_settings__get_allow_remote(the_repository) > 0) {
> > >>> + strbuf_reset(&tmp); strbuf_realpath_forgiving(&tmp, paths[k],
> > >>> + 0); path_k = tmp.buf; } else path_k = paths[k];
> > >>
> > >> This feels wrong.
> > >>
> > >> It is not that fsmonitor.allowRemote is true, but whether or not
> > >> this particular file system *IS* remote, right?
> > >
> > > True. I suppose each path could be checked, or some bit could be set
> > > if the working directory is remote, perhaps along the lines of
> > > fsmonitor_ipc__get_path. Then the determination of the IPC path and
> > > whether it is remote would be in one place. fsm-settings-* would
> > > then just get that bit and check it against allowRemote.
> > >
> > > Thoughts?
> >
> > I'll do some digging here. There ought to be a way to tell if a
> > component directory in a pathname has a firmlink peer. (But I'm
> > traveling for GitMerge, so I might not have time to look at this until
> > afterwards.)
> >
> > This would indicate that a "bi-directional wormhole" (their words)
> > alias is available (and hopefully, a way to computer the other
> > peer....)
> >
> > I'm thinking that the "/network/..." path in the FSEvents is because
> > FSEventD is using a particular peer spelling (that we weren't
> > expecting).
> >
> > If we can compute the spellings of the peers once when the daemon
> > starts up (and maybe make a little dictionary), then we can do prefix
> > tricks on the absolute path after the path_k = paths[k]; step in
> > fsevent_callback() to extract a relative path rather than an absolute
> > path.
> >
> > Then call fsmonitor_classify_path_relative() instead of _absolute().
> >
> > Watch out though, there are several places that do rel = path+k +
> > ...state->path_worktree_watch.len...
> > that would need to be adjusted.
> >
> > Hope this helps,
> > Jeff
> >
> >
> Thanks for the insights, I'll dig around and test things out some more.
> 
> -Eric

This is informative:

https://developer.apple.com/forums/thread/120665

-Eric



^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-06 19:33                 ` Eric DeCosta
@ 2022-09-06 22:11                   ` Eric DeCosta
  2022-09-07 19:14                   ` Jeff Hostetler
  1 sibling, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-06 22:11 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Eric DeCosta
> Sent: Tuesday, September 6, 2022 3:34 PM
> To: 'Jeff Hostetler' <git@jeffhostetler.com>; 'Eric DeCosta via GitGitGadget'
> <gitgitgadget@gmail.com>; 'git@vger.kernel.org' <git@vger.kernel.org>
> Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the
> real path
> 
> 
> 
> > -----Original Message-----
> > From: Eric DeCosta
> > Sent: Tuesday, September 6, 2022 3:02 PM
> > To: Jeff Hostetler <git@jeffhostetler.com>; Eric DeCosta via
> > GitGitGadget <gitgitgadget@gmail.com>; git@vger.kernel.org
> > Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths
> > to the real path
> >
> >
> >
> > > -----Original Message-----
> > > From: Jeff Hostetler <git@jeffhostetler.com>
> > > Sent: Tuesday, September 6, 2022 1:14 PM
> > > To: Eric DeCosta <edecosta@mathworks.com>; Eric DeCosta via
> > > GitGitGadget <gitgitgadget@gmail.com>; git@vger.kernel.org
> > > Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event
> > > paths to the real path
> > >
> > >
> > >
> > > On 9/2/22 12:35 PM, Eric DeCosta wrote:
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Jeff Hostetler <git@jeffhostetler.com>
> > > >> Sent: Thursday, September 1, 2022 4:06 PM
> > > >> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> > > >> git@vger.kernel.org
> > > >> Cc: Eric DeCosta <edecosta@mathworks.com>
> > > >> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event
> > > >> paths to the real path
> > > >>
> > > >>
> > > >>
> > > >> On 8/31/22 12:09 PM, Eric DeCosta via GitGitGadget wrote:
> > > >>> From: Eric DeCosta <edecosta@mathworks.com>
> > > >>>
> > > >>> Consider the following network working directory that is mounted
> > > >>> under
> > > >>> /System/Volumes/Data:
> > > >>>
> > > >>> /network/working/directory
> > > >>>
> > > >>> The git working directory path is:
> > > >>>
> > > >>> /System/Volumes/Data/network/working/directory
> > > >>>
> > > >>> The paths reported by FSEvents always start with /network.
> > > >>> fsmonitor expects paths to be under the working directory;
> > > >>> therefore it fails to match /network/... and ignores the change.
> > > >>
> > > >> I'm not sure I understand what's going on here.
> > > >> Are you saying that FSEvents reports network mounted directories
> > > >> with a path relative to the mount-point rather than an absolute path?
> > > >>
> > > >
> > > > Yes
> > > >
> > > >> In your example, is "/network/working/directory" a valid path on
> > > >> your system (that also happens to be the same directory as
> > > >> "/System/Volumes/...")
> > > >>
> > > >> That is, do you have some aliasing going on where both paths are
> > > >> valid -- like a pair of hard-linked directories?
> > > >> Or, is there something special about a network mount point?
> > > >>
> > > >>
> > > >> Did you have to "cd /System/Volumes/..." to get Git to have the
> > > >> absolute path be this? Or were you doing a "cd /network/..."?
> > > >> (Sorry to ask so many questions but I don't have a pair of
> > > >> systems to test any of this on right now.)
> > > >>
> > > >
> > > > "/network/working/directory" is mounted under
> > > > "/System/Volumes/Data/network/working/directory"
> > > >
> > > > There is also a symlink:
> > > >
> > > > "/network" -> "/System/Volumes/Data/network"
> > > >
> > > > No matter if I "cd /System/Volumes/Data/network/working/directory"
> > > > or "cd /network/working/directory" the paths reported by FSEvents
> > > > always start with "/network/working/directory"
> > > >
> > > > If I do a similar symlink with local directories, I do not get
> > > > this unexpected behavior.
> > > >
> > > > I need to remove the symlink and try it that way, but I need to
> > > > coordinate with the machine's owner.
> > >
> > >
> > > I think you have stumbled upon a recent MacOS feature called
> "firmlinks".
> > > I'm just reading up on it myself, so I shouldn't speculate here
> > > (yet), but maybe you too could do some reading on the subject.
> > >
> > > This makes me wonder if the symlink is historical. The real magic is
> > > in the firmlinks. For example, if I do:
> > >
> > > $ (cd / ; ls -l | grep Users)
> > > drwxr-xr-x 6 root admin 192 Apr 6 2020 Users
> > >
> > > $ (cd /Users ; df .)
> > > Filesystem 512-blocks Used Available Capacity iused ifree %iused
> > > Mounted on
> > > /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557
> 0%
> > > /System/Volumes/Data
> > >
> > > we can see that /Users is on /System/Volumes/Data (and there is a
> > > /System/Volumes/Data/Users directory with the same metadata), but it
> > > is not a symlink.
> > >
> > >
> > > See [1] for more info.
> > >
> > > [1]
> > > http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
> > > magic.html <https://protect-
> > >
> >
> us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co
> > > m>
> > >
> > >
> >
> > Hmm, I don't see anything that suggests a firmlink is involved but
> > this is new to me so maybe I just don't see it. There is an automount
> > in addition to the symlink though.
> >
> > So, dispensing with the abstract "/network" path, and getting to the
> > details of my environment:
> >
> > % mount | grep /mathworks
> > map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks
> (autofs,
> > automounted, nobrowse) triggered map auto.sfs.sol2.devel on
> > /System/Volumes/Data/mathworks/devel (autofs, automounted,
> > nobrowse) triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on
> > /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted,
> > nobrowse) triggered map sfs.worldwide.US-Natick.devel.sbs.29 on
> > /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted,
> > nobrowse)
> > batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on
> >
> /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
> > 28560.2 (nfs, nodev, automounted, noatime, nobrowse) ...
> >
> > % ls -l / | grep mathworks
> > lrwxr-xr-x   1 root  wheel    35 Aug 29 10:25 home@ ->
> > /System/Volumes/Data/mathworks/home
> > lrwxr-xr-x   1 root  wheel    30 Aug 29 10:25 mathworks@ ->
> > /System/Volumes/Data/mathworks
> >
> > My worktree is:
> >
> /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
> > 28560.2
> > or, equivalently via the symlink just:
> > /mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2
> >
> > I have sudo'ers access now; so I can try mounting things manually,
> > mess about with symlinks and see what I get.
> >
> > > [...]
> > > >>> @@ -209,7 +209,12 @@ static void
> > > >> fsevent_callback(ConstFSEventStreamRef streamRef,
> > > >>> /*
> > > >>> * On Mac, we receive an array of absolute paths.
> > > >>> */
> > > >>> - path_k = paths[k];
> > > >>> + if (fsm_settings__get_allow_remote(the_repository) > 0) {
> > > >>> + strbuf_reset(&tmp); strbuf_realpath_forgiving(&tmp, paths[k],
> > > >>> + 0); path_k = tmp.buf; } else path_k = paths[k];
> > > >>
> > > >> This feels wrong.
> > > >>
> > > >> It is not that fsmonitor.allowRemote is true, but whether or not
> > > >> this particular file system *IS* remote, right?
> > > >
> > > > True. I suppose each path could be checked, or some bit could be
> > > > set if the working directory is remote, perhaps along the lines of
> > > > fsmonitor_ipc__get_path. Then the determination of the IPC path
> > > > and whether it is remote would be in one place. fsm-settings-*
> > > > would then just get that bit and check it against allowRemote.
> > > >
> > > > Thoughts?
> > >
> > > I'll do some digging here. There ought to be a way to tell if a
> > > component directory in a pathname has a firmlink peer. (But I'm
> > > traveling for GitMerge, so I might not have time to look at this
> > > until
> > > afterwards.)
> > >
> > > This would indicate that a "bi-directional wormhole" (their words)
> > > alias is available (and hopefully, a way to computer the other
> > > peer....)
> > >
> > > I'm thinking that the "/network/..." path in the FSEvents is because
> > > FSEventD is using a particular peer spelling (that we weren't
> > > expecting).
> > >
> > > If we can compute the spellings of the peers once when the daemon
> > > starts up (and maybe make a little dictionary), then we can do
> > > prefix tricks on the absolute path after the path_k = paths[k]; step
> > > in
> > > fsevent_callback() to extract a relative path rather than an
> > > absolute path.
> > >
> > > Then call fsmonitor_classify_path_relative() instead of _absolute().
> > >
> > > Watch out though, there are several places that do rel = path+k +
> > > ...state->path_worktree_watch.len...
> > > that would need to be adjusted.
> > >
> > > Hope this helps,
> > > Jeff
> > >
> > >
> > Thanks for the insights, I'll dig around and test things out some more.
> >
> > -Eric
> 
> This is informative:
> 
> https://developer.apple.com/forums/thread/120665
> 
> -Eric
> 
Conversation with one of our macOS admins:

Q: Is /mathworks a symbolic link or some sort of firmlink or synthetic firmlink?

A: It’s a “synthetic” link, per apple. Config is managed in /etc/synthetic.conf

The auto mounter will also automatically create it based on the contents of /etc/auto_master even without the synthetic.conf file. We went back and forth with apple for a while on the “right” way to do it and now I’m not sure what we settled on, off the top of my head

I don’t know if they’re the same under the hood, but it seemed to be able to create root level links even though you can’t with ln anymore

I suspect apple wants users to use the synthetic.conf file in all cases...
----

It appears that the automounter is creating "synthetic firmlinks", a la https://derflounder.wordpress.com/2020/01/18/creating-root-level-directories-and-symbolic-links-on-macos-catalina/

When I look in /etc/auto_master I see several entries that appear as symlinks in the output of "ls -l /" but in reality they are synthetic firmlinks.

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-06 19:33                 ` Eric DeCosta
  2022-09-06 22:11                   ` Eric DeCosta
@ 2022-09-07 19:14                   ` Jeff Hostetler
  2022-09-07 23:04                     ` Eric DeCosta
  1 sibling, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-07 19:14 UTC (permalink / raw)
  To: Eric DeCosta, Eric DeCosta via GitGitGadget, git



On 9/6/22 3:33 PM, Eric DeCosta wrote:
> 
> 
[...]
> 
> This is informative:
> 
> https://developer.apple.com/forums/thread/120665
> 
> -Eric
> 
> 

How strange........

Have you tried the:
    fcntl(F_GETPATH)
vs fcntl(F_GETPATH_NOFIRMLINK)
vs realpath()

comparison suggested in the comments and does it return
anything sensical?  Such that we could record both spellings
when the daemon starts up.  I think we'd have to do it on
an open fd on the worktree root directory.

Jeff


https://developer.apple.com/forums/thread/120665

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
  2022-09-07 19:14                   ` Jeff Hostetler
@ 2022-09-07 23:04                     ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-07 23:04 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Wednesday, September 7, 2022 3:15 PM
> To: Eric DeCosta <edecosta@mathworks.com>; Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com>; git@vger.kernel.org
> Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the
> real path
> 
> 
> 
> On 9/6/22 3:33 PM, Eric DeCosta wrote:
> >
> >
> [...]
> >
> > This is informative:
> >
> > https://developer.apple.com/forums/thread/120665
> > <https://protect-
> us.mimecast.com/s/Vj1mCL9GlLUjmyJjfqLBqG?domain=devel
> > oper.apple.com>
> >
> > -Eric
> >
> >
> 
> How strange........
> 
> Have you tried the:
> fcntl(F_GETPATH)
> vs fcntl(F_GETPATH_NOFIRMLINK)
> vs realpath()
> 
> comparison suggested in the comments and does it return anything sensical?
> Such that we could record both spellings when the daemon starts up. I think
> we'd have to do it on an open fd on the worktree root directory.
> 
> Jeff
> 
> 
> https://developer.apple.com/forums/thread/120665 <https://protect-
> us.mimecast.com/s/Vj1mCL9GlLUjmyJjfqLBqG?domain=developer.apple.com
> >

I'm not getting the same result:

Worktree is: '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
F_GETPATH of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
F_GETPATH_NOFIRMLINK of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
realpath() of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'

Trying /mathworks/devel/sandbox/edecosta/git
F_GETPATH of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
F_GETPATH_NOFIRMLINK of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
realpath() of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'

Either something has changed or the fact that "/mathworks" is a "synthetic firmlink" has something to do with it.

Maybe look for synthetic firmlinks in the root directory and use that information to figure out what aliases there might be to the worktree.

lrwxr-xr-x   1 root  wheel    30 Sep  2 11:52 mathworks -> /System/Volumes/Data/mathworks

If what the link resolves to is a prefix of the worktree, then it is a firmlink to the worktree.

Something like that.

-Eric

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-08-31 16:09     ` [PATCH v4 0/4] " Eric DeCosta via GitGitGadget
                         ` (3 preceding siblings ...)
  2022-08-31 16:09       ` [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path Eric DeCosta via GitGitGadget
@ 2022-09-10 20:00       ` Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 1/4] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                           ` (6 more replies)
  4 siblings, 7 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-10 20:00 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

Eric DeCosta (4):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS

 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |   8 ++
 compat/fsmonitor/fsm-ipc-darwin.c        |  46 ++++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   4 +
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 106 ++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  70 ++++--------
 compat/fsmonitor/fsm-settings-win32.c    | 106 +-----------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   6 ++
 fsmonitor-ipc.c                          |   2 -
 fsmonitor-path-utils.h                   |  49 +++++++++
 fsmonitor-settings.c                     |  58 +++++++++-
 fsmonitor-settings.h                     |   2 +-
 15 files changed, 441 insertions(+), 159 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: be1a02a17ede4082a86dfbfee0f54f345e8b43ac
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v4:

 -:  ----------- > 1:  2f647b53e4d fsmonitor: refactor filesystem checks to common interface
 2:  2cb026a6317 ! 2:  28d08bcf808 fsmonitor: generate unique Unix socket file name in the desired location
     @@ Metadata
      Author: Eric DeCosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    fsmonitor: generate unique Unix socket file name in the desired location
     +    fsmonitor: relocate socket file if .git directory is remote
      
     -    Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir
     -    locate the Unix domain socket file in the desired location (either
     -    the .git directory, $HOME, or fsmonitor.socketDir). If the location
     -    is other than the .git directory, generate a unique file name based
     -    on the SHA1 has of the path to the .git directory.
     +    If the .git directory is on a remote file system, create the socket
     +    file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     - ## fsmonitor-ipc.c ##
     -@@
     - #include "cache.h"
     --#include "fsmonitor.h"
     --#include "simple-ipc.h"
     - #include "fsmonitor-ipc.h"
     -+#include "fsmonitor-settings.h"
     - #include "run-command.h"
     - #include "strbuf.h"
     - #include "trace2.h"
     -@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
     - 	return 1;
     - }
     + ## Makefile ##
     +@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND
     + 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
     + 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
     + 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
     ++	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
     + endif
       
     --GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     -+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
     + ifdef FSMONITOR_OS_SETTINGS
     +
     + ## compat/fsmonitor/fsm-ipc-darwin.c (new) ##
     +@@
     ++#include "cache.h"
     ++#include "config.h"
     ++#include "strbuf.h"
     ++#include "fsmonitor.h"
     ++#include "fsmonitor-ipc.h"
     ++#include "fsmonitor-path-utils.h"
     ++
     ++static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
      +
      +const char *fsmonitor_ipc__get_path(void)
      +{
     -+#ifdef WIN32
     -+	return fsmonitor_ipc__get_default_path();
     -+#else
     -+	char *retval;
     ++	static const char *ipc_path;
      +	SHA_CTX sha1ctx;
     -+	const char *git_dir;
     -+	const char *sock_dir;
     ++	char *sock_dir;
      +	struct strbuf ipc_file = STRBUF_INIT;
      +	unsigned char hash[SHA_DIGEST_LENGTH];
      +
     -+	if (fsm_settings__get_allow_remote(the_repository) < 1)
     -+		return fsmonitor_ipc__get_default_path();
     ++	if (ipc_path)
     ++		return ipc_path;
      +
     -+	git_dir = get_git_dir();
     -+	sock_dir = fsm_settings__get_socket_dir(the_repository);
     ++	ipc_path = fsmonitor_ipc__get_default_path();
     ++
     ++	/* By default the socket file is created in the .git directory */
     ++	if (fsmonitor__is_fs_remote(ipc_path) < 1)
     ++		return ipc_path;
      +
      +	SHA1_Init(&sha1ctx);
     -+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
     ++	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
      +	SHA1_Final(hash, &sha1ctx);
      +
     ++	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
     ++
     ++	/* Create the socket file in either socketDir or $HOME */
      +	if (sock_dir && *sock_dir)
      +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
      +					sock_dir, hash_to_hex(hash));
      +	else
      +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
     -+	retval = interpolate_path(ipc_file.buf, 1);
     -+	if (!retval)
     ++
     ++	ipc_path = interpolate_path(ipc_file.buf, 1);
     ++	if (!ipc_path)
      +		die(_("Invalid path: %s"), ipc_file.buf);
     ++
      +	strbuf_release(&ipc_file);
     -+	return retval;
     -+#endif
     ++	return ipc_path;
      +}
     +
     + ## compat/fsmonitor/fsm-ipc-win32.c (new) ##
     +@@
     ++#include "cache.h"
     ++#include "fsmonitor-ipc.h"
     ++
     ++GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     +
     + ## contrib/buildsystems/CMakeLists.txt ##
     +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
     + 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
     ++		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
       
     - enum ipc_active_state fsmonitor_ipc__get_state(void)
     - {
     + 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
     +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
     + 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
     ++		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
     + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
     + 
     + 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
      
     - ## fsmonitor-ipc.h ##
     -@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
     -  */
     - const char *fsmonitor_ipc__get_path(void);
     + ## fsmonitor-ipc.c ##
     +@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
     + 	return 1;
     + }
       
     -+/*
     -+ * Returns the pathname to the default IPC named pipe or Unix domain
     -+ * socket.
     -+ */
     -+const char *fsmonitor_ipc__get_default_path(void);
     -+
     - /*
     -  * Try to determine whether there is a `git-fsmonitor--daemon` process
     -  * listening on the IPC pipe/socket.
     +-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     +-
     + enum ipc_active_state fsmonitor_ipc__get_state(void)
     + {
     + 	return ipc_get_active_state(fsmonitor_ipc__get_path());
 1:  836a791e6b7 ! 3:  ff54b6e0bb5 fsmonitor: add two new config options, allowRemote and socketDir
     @@ Metadata
      Author: Eric DeCosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    fsmonitor: add two new config options, allowRemote and socketDir
     +    fsmonitor: avoid socket location check if using hook
      
     -    Introduce two new configuration options
     -
     -       fsmonitor.allowRemote - setting this to true overrides fsmonitor's
     -       default behavior of erroring out when enountering network file
     -       systems. Additionly, when true, the Unix domain socket (UDS) file
     -       used for IPC is located in $HOME rather than in the .git directory.
     -
     -       fsmonitor.socketDir - allows for the UDS file to be located
     -       anywhere the user chooses rather $HOME.
     +    If monitoring is done via fsmonitor hook rather than IPC there is no
     +    need to check if the location of the Unix Domain socket (UDS) file is
     +    on a remote filesystem.
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     - ## fsmonitor-settings.c ##
     -@@
     - struct fsmonitor_settings {
     - 	enum fsmonitor_mode mode;
     + ## compat/fsmonitor/fsm-settings-darwin.c ##
     +@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_uds_volume(struct repository *r)
     + 	return FSMONITOR_REASON_OK;
     + }
     + 
     +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
     ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
     + {
       	enum fsmonitor_reason reason;
     -+	int allow_remote;
     - 	char *hook_path;
     -+	char *sock_dir;
     - };
       
     - static enum fsmonitor_reason check_for_incompatible(struct repository *r)
     -@@ fsmonitor-settings.c: static struct fsmonitor_settings *alloc_settings(void)
     - 	CALLOC_ARRAY(s, 1);
     - 	s->mode = FSMONITOR_MODE_DISABLED;
     - 	s->reason = FSMONITOR_REASON_UNTESTED;
     -+	s->allow_remote = -1;
     +-	reason = check_uds_volume(r);
     +-	if (reason != FSMONITOR_REASON_OK)
     +-		return reason;
     ++	if (ipc) {
     ++		reason = check_uds_volume(r);
     ++		if (reason != FSMONITOR_REASON_OK)
     ++			return reason;
     ++	}
       
     - 	return s;
     + 	return FSMONITOR_REASON_OK;
       }
     -@@ fsmonitor-settings.c: static void lookup_fsmonitor_settings(struct repository *r)
     - 		fsm_settings__set_disabled(r);
     +
     + ## compat/fsmonitor/fsm-settings-win32.c ##
     +@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4git(struct repository *r)
     + 	return FSMONITOR_REASON_OK;
       }
       
     -+int fsm_settings__get_allow_remote(struct repository *r)
     -+{
     -+	if (!r)
     -+		r = the_repository;
     -+	if (!r->settings.fsmonitor)
     -+		lookup_fsmonitor_settings(r);
     -+
     -+	return r->settings.fsmonitor->allow_remote;
     -+}
     -+
     -+const char *fsm_settings__get_socket_dir(struct repository *r)
     -+{
     -+	if (!r)
     -+		r = the_repository;
     -+	if (!r->settings.fsmonitor)
     -+		lookup_fsmonitor_settings(r);
     -+
     -+	return r->settings.fsmonitor->sock_dir;
     -+}
     -+
     - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
     +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
     ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
       {
     - 	if (!r)
     -@@ fsmonitor-settings.c: enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
     - 	return r->settings.fsmonitor->mode;
     + 	enum fsmonitor_reason reason;
     + 
     +
     + ## fsmonitor-settings.c ##
     +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_remote(struct repository *r)
       }
     + #endif
       
     -+
     - const char *fsm_settings__get_hook_path(struct repository *r)
     +-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
     ++static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
       {
     - 	if (!r)
     + 	if (!r->worktree) {
     + 		/*
     +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_for_incompatible(struct repository *r)
     + 		reason = check_remote(r);
     + 		if (reason != FSMONITOR_REASON_OK)
     + 			return reason;
     +-		reason = fsm_os__incompatible(r);
     ++		reason = fsm_os__incompatible(r, ipc);
     + 		if (reason != FSMONITOR_REASON_OK)
     + 			return reason;
     + 	}
      @@ fsmonitor-settings.c: const char *fsm_settings__get_hook_path(struct repository *r)
     - 	return r->settings.fsmonitor->hook_path;
     - }
       
     -+void fsm_settings__set_allow_remote(struct repository *r)
     -+{
     -+	int allow;
     -+
     -+	if (!r)
     -+		r = the_repository;
     -+	if (!r->settings.fsmonitor)
     -+		r->settings.fsmonitor = alloc_settings();
     -+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
     -+		r->settings.fsmonitor->allow_remote = allow;
     -+
     -+	return;
     -+}
     -+
     -+void fsm_settings__set_socket_dir(struct repository *r)
     -+{
     -+	const char *path;
     -+
     -+	if (!r)
     -+		r = the_repository;
     -+	if (!r->settings.fsmonitor)
     -+		r->settings.fsmonitor = alloc_settings();
     -+
     -+	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
     -+		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
     -+		r->settings.fsmonitor->sock_dir = strdup(path);
     -+	}
     -+
     -+	return;
     -+}
     -+
       void fsm_settings__set_ipc(struct repository *r)
       {
      -	enum fsmonitor_reason reason = check_for_incompatible(r);
     -+	enum fsmonitor_reason reason;
     -+
     -+	fsm_settings__set_allow_remote(r);
     -+	fsm_settings__set_socket_dir(r);
     -+	reason = check_for_incompatible(r);
     ++	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
       
       	if (reason != FSMONITOR_REASON_OK) {
       		fsm_settings__set_incompatible(r, reason);
     @@ fsmonitor-settings.c: void fsm_settings__set_ipc(struct repository *r)
       void fsm_settings__set_hook(struct repository *r, const char *path)
       {
      -	enum fsmonitor_reason reason = check_for_incompatible(r);
     -+	enum fsmonitor_reason reason;
     -+
     -+	fsm_settings__set_allow_remote(r);
     -+	fsm_settings__set_socket_dir(r);
     -+	reason = check_for_incompatible(r);
     ++	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
       
       	if (reason != FSMONITOR_REASON_OK) {
       		fsm_settings__set_incompatible(r, reason);
      
       ## fsmonitor-settings.h ##
     -@@ fsmonitor-settings.h: enum fsmonitor_reason {
     - 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
     - };
     - 
     -+void fsm_settings__set_allow_remote(struct repository *r);
     -+void fsm_settings__set_socket_dir(struct repository *r);
     - void fsm_settings__set_ipc(struct repository *r);
     - void fsm_settings__set_hook(struct repository *r, const char *path);
     - void fsm_settings__set_disabled(struct repository *r);
     - void fsm_settings__set_incompatible(struct repository *r,
     - 				    enum fsmonitor_reason reason);
     - 
     -+int fsm_settings__get_allow_remote(struct repository *r);
     -+const char *fsm_settings__get_socket_dir(struct repository *r);
     - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
     - const char *fsm_settings__get_hook_path(struct repository *r);
     +@@ fsmonitor-settings.h: struct fsmonitor_settings;
     +  * fsm_os__* routines should considered private to fsm_settings__
     +  * routines.
     +  */
     +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
     ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
     + #endif /* HAVE_FSMONITOR_OS_SETTINGS */
       
     + #endif /* FSMONITOR_SETTINGS_H */
 3:  a3110f1e25a < -:  ----------- fsmonitor: ensure filesystem and unix socket filesystem are compatible
 4:  56cabf3be3b ! 4:  b7d6cf44695 fsmonitor: normalize FSEvents event paths to the real path
     @@ Metadata
      Author: Eric DeCosta <edecosta@mathworks.com>
      
       ## Commit message ##
     -    fsmonitor: normalize FSEvents event paths to the real path
     +    fsmonitor: deal with synthetic firmlinks on macOS
      
     -    Consider the following network working directory that is mounted under
     -    /System/Volumes/Data:
     +    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
     +    called 'firmlinks' in order to separate the boot volume into two
     +    volumes, one read-only and one writable but still present them to the
     +    user as a single volume. Along with this change, Apple removed the
     +    ability to create symlinks in the root directory and replaced them with
     +    'synthetic firmlinks'. See 'man synthetic.conf'
      
     -    /network/working/directory
     +    When FSEevents reports the path of changed files, if the path invloves
     +    a synthetic firmlink, the path is reported from the point of the
     +    synthetic firmlink and not the real path. For example:
      
     -    The git working directory path is:
     +    Real path:
     +    /System/Volumes/Data/network/working/directory/foo.txt
      
     -    /System/Volumes/Data/network/working/directory
     +    Synthetic firmlink:
     +    /network -> /System/Volumes/Data/network
      
     -    The paths reported by FSEvents always start with /network. fsmonitor
     -    expects paths to be under the working directory; therefore it
     -    fails to match /network/... and ignores the change.
     +    FSEvents path:
     +    /network/working/directory/foo.txt
      
     -    Change things such that if fsmonitor.allowRemote is true that the
     -    paths reported via FSEevents are normalized to the real path.
     +    This causes the FSEvents path to not match against the worktree
     +    directory.
     +
     +    There are several ways in which synthetic firmlinks can be created:
     +    they can be defined in /etc/synthetic.conf, the automounter can create
     +    them, and there may be other means. Simply reading /etc/synthetic.conf
     +    is insufficient. No matter what process creates synthetic firmlinks,
     +    they all get created in the root directory.
     +
     +    Therefore, in order to deal with synthetic firmlinks, the root directory
     +    is scanned and the first possible synthetic firmink that, when resolved,
     +    is a prefix of the worktree is used to map FSEvents paths to worktree
     +    paths.
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     + ## builtin/fsmonitor--daemon.c ##
     +@@
     + #include "parse-options.h"
     + #include "fsmonitor.h"
     + #include "fsmonitor-ipc.h"
     ++#include "fsmonitor-path-utils.h"
     + #include "compat/fsmonitor/fsm-health.h"
     + #include "compat/fsmonitor/fsm-listen.h"
     + #include "fsmonitor--daemon.h"
     +@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
     + 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
     + 	state.nr_paths_watching = 1;
     + 
     ++	strbuf_init(&state.alias.alias, 0);
     ++	strbuf_init(&state.alias.points_to, 0);
     ++	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
     ++		err = error(_("could not get worktree alias"));
     ++		goto done;
     ++	}
     ++
     + 	/*
     + 	 * We create and delete cookie files somewhere inside the .git
     + 	 * directory to help us keep sync with the file system.  If
     +
       ## compat/fsmonitor/fsm-listen-darwin.c ##
      @@
       #include "fsmonitor.h"
       #include "fsm-listen.h"
       #include "fsmonitor--daemon.h"
     -+#include "fsmonitor-settings.h"
     ++#include "fsmonitor-path-utils.h"
       
       struct fsm_listen_data
       {
     -@@ compat/fsmonitor/fsm-listen-darwin.c: static void my_add_path(struct fsmonitor_batch *batch, const char *path)
     - 	free(composed);
     - }
     - 
     --
     - static void fsevent_callback(ConstFSEventStreamRef streamRef,
     - 			     void *ctx,
     - 			     size_t num_of_events,
      @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
       		/*
       		 * On Mac, we receive an array of absolute paths.
       		 */
      -		path_k = paths[k];
     -+		if (fsm_settings__get_allow_remote(the_repository) > 0) {
     -+			strbuf_reset(&tmp);
     -+			strbuf_realpath_forgiving(&tmp, paths[k], 0);
     -+			path_k = tmp.buf;
     -+		} else
     ++		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
     ++		if (!path_k)
      +			path_k = paths[k];
       
       		/*
       		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
      @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
     - 
       			fsmonitor_force_resync(state);
       			fsmonitor_batch__free_list(batch);
     -+			batch = NULL;
       			string_list_clear(&cookie_list, 0);
     ++			batch = NULL;
       
       			/*
     -@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
     + 			 * We assume that any events that we received
     +
     + ## compat/fsmonitor/fsm-path-utils-darwin.c ##
     +@@
     + #include "fsmonitor.h"
     + #include "fsmonitor-path-utils.h"
     ++#include <dirent.h>
     ++#include <errno.h>
     ++#include <fcntl.h>
     + #include <sys/param.h>
     + #include <sys/mount.h>
     + 
     +@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char *path)
     + 		return -1;
     + 	return fs.is_remote;
     + }
     ++
     ++/*
     ++ * Scan the root directory for synthetic firmlinks that when resolved
     ++ * are a prefix of the path, stopping at the first one found.
     ++ *
     ++ * Some information about firmlinks and synthetic firmlinks:
     ++ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
     ++ *
     ++ * macOS no longer allows symlinks in the root directory; any link found
     ++ * there is therefore a synthetic firmlink.
     ++ *
     ++ * If this function gets called often, will want to cache all the firmlink
     ++ * information, but for now there is only one caller of this function.
     ++ *
     ++ * If there is more than one alias for the path, that is another
     ++ * matter altogteher.
     ++ */
     ++int fsmonitor__get_alias(const char *path, struct alias_info *info)
     ++{
     ++	DIR * dir;
     ++	int read;
     ++	int retval;
     ++	struct dirent *de;
     ++	struct strbuf alias;
     ++	struct strbuf points_to;
     ++
     ++	retval = 0;
     ++	dir = opendir("/");
     ++	if (!dir)
     ++		return -1;
     ++
     ++	strbuf_init(&alias, 256);
     ++	strbuf_init(&points_to, MAXPATHLEN);
     ++
     ++	while ((de = readdir(dir)) != NULL) {
     ++		strbuf_reset(&alias);
     ++		strbuf_addch(&alias, '/');
     ++		strbuf_add(&alias, de->d_name, strlen(de->d_name));
     ++
     ++		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
     ++		if (read > 0) {
     ++			strbuf_setlen(&points_to, read);
     ++			if ((strncmp(points_to.buf, path, points_to.len) == 0)
     ++				&& path[points_to.len] == '/') {
     ++				strbuf_addbuf(&info->alias, &alias);
     ++				strbuf_addbuf(&info->points_to, &points_to);
     ++				trace_printf_key(&trace_fsmonitor,
     ++					"Found alias for '%s' : '%s' -> '%s'",
     ++					path, info->alias.buf, info->points_to.buf);
     ++				retval = 0;
     ++				goto done;
     ++			}
     ++		} else if (errno != EINVAL) { /* Something other than not a link */
     ++			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
     ++			retval = -1;
     ++			goto done;
     ++		}
     ++	}
     ++
     ++	done:
     ++	closedir(dir);
     ++	strbuf_release(&alias);
     ++	strbuf_release(&points_to);
     ++	return retval;
     ++}
     ++
     ++char *fsmonitor__resolve_alias(const char *path,
     ++	const struct alias_info *info)
     ++{
     ++	int len = info->alias.len;
     ++
     ++	if (!len)
     ++		return NULL;
     ++
     ++	if ((strncmp(info->alias.buf, path, len) == 0)
     ++		&& path[len] == '/') {
     ++		struct strbuf tmp;
     ++		const char *remainder = path + len;
     ++		int rem_len = strlen(remainder);
     ++
     ++		strbuf_init(&tmp, info->points_to.len + rem_len);
     ++		strbuf_addbuf(&tmp, &info->points_to);
     ++		strbuf_add(&tmp, remainder, rem_len);
     ++		return strbuf_detach(&tmp, NULL);
     ++	}
     ++
     ++	return NULL;
     ++}
     +
     + ## compat/fsmonitor/fsm-path-utils-win32.c ##
     +@@ compat/fsmonitor/fsm-path-utils-win32.c: int fsmonitor__is_fs_remote(const char *path)
     + 		return -1;
     + 	return fs.is_remote;
     + }
     ++
     ++/*
     ++ * No-op for now.
     ++ */
     ++int fsmonitor__get_alias(const char *path, struct alias_info *info)
     ++{
     ++	return 0;
     ++}
     ++
     ++/*
     ++ * No-op for now.
     ++ */
     ++char *fsmonitor__resolve_alias(const char *path,
     ++	const struct alias_info *info)
     ++{
     ++	return NULL;
     ++}
     +
     + ## fsmonitor--daemon.h ##
     +@@
     + #include "run-command.h"
     + #include "simple-ipc.h"
     + #include "thread-utils.h"
     ++#include "fsmonitor-path-utils.h"
     + 
     + struct fsmonitor_batch;
     + struct fsmonitor_token_data;
     +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
     + 
     + 	struct strbuf path_worktree_watch;
     + 	struct strbuf path_gitdir_watch;
     ++	struct alias_info alias;
     + 	int nr_paths_watching;
     + 
     + 	struct fsmonitor_token_data *current_token_data;
     +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
     + 
     + 	struct ipc_server_data *ipc_server_data;
     + 	struct strbuf path_ipc;
     ++
     + };
     + 
     + /*
     +@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
     +  */
     + void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
     + 
     ++char *fsmonitor_resolve_alias(const char *path,
     ++	struct alias_info *alias);
     ++
     + #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
     + #endif /* FSMONITOR_DAEMON_H */
     +
     + ## fsmonitor-path-utils.h ##
     +@@
     + #ifndef FSM_PATH_UTILS_H
     + #define FSM_PATH_UTILS_H
       
     - 		case IS_WORKDIR_PATH:
     - 			/* try to queue normal pathnames */
     --
     - 			if (trace_pass_fl(&trace_fsmonitor))
     - 				log_flags_set(path_k, event_flags[k]);
     ++#include "strbuf.h"
     ++
     ++struct alias_info
     ++{
     ++	struct strbuf alias;
     ++	struct strbuf points_to;
     ++};
     ++
     + struct fs_info {
     + 	int is_remote;
     + 	char *typename;
     +@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
     +  */
     + int fsmonitor__is_fs_remote(const char *path);
       
     ++/*
     ++ * Get the alias in given path, if any.
     ++ *
     ++ * Sets alias to the first alias that matches any part of the path.
     ++ *
     ++ * Returns -1 on error, 0 otherwise.
     ++ */
     ++int fsmonitor__get_alias(const char *path, struct alias_info *info);
     ++
     ++/*
     ++ * Resolve the path against the given alias.
     ++ *
     ++ * Returns the resolved path if there is one, NULL otherwise.
     ++ */
     ++char *fsmonitor__resolve_alias(const char *path,
     ++	const struct alias_info *info);
     ++
     ++
     + #endif

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v5 1/4] fsmonitor: refactor filesystem checks to common interface
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
@ 2022-09-10 20:00         ` Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 2/4] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
                           ` (5 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-10 20:00 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Provide a common interface for getting basic filesystem information
including filesystem type and whether the filesystem is remote.

Refactor existing code for getting basic filesystem info and detecting
remote file systems to the new interface.

Refactor filesystem checks to leverage new interface. For macOS,
error-out if the Unix Domain socket (UDS) file is on a remote
filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                                 |   1 +
 compat/fsmonitor/fsm-path-utils-darwin.c |  40 +++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  |  89 +++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  62 ++++----------
 compat/fsmonitor/fsm-settings-win32.c    | 104 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   2 +
 fsmonitor-path-utils.h                   |  23 +++++
 fsmonitor-settings.c                     |  50 +++++++++++
 8 files changed, 224 insertions(+), 147 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h

diff --git a/Makefile b/Makefile
index 924b864ae83..265fc585286 100644
--- a/Makefile
+++ b/Makefile
@@ -2042,6 +2042,7 @@ endif
 ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
 ifeq ($(TCLTK_PATH),)
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
new file mode 100644
index 00000000000..067cbe6990a
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -0,0 +1,40 @@
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+#include <sys/param.h>
+#include <sys/mount.h>
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	struct statfs fs;
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return -1;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
+	if (!(fs.f_flags & MNT_LOCAL))
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	fs_info->typename = fs.f_fstypename;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
new file mode 100644
index 00000000000..dc211de514f
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -0,0 +1,89 @@
+#include "cache.h"
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+
+/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
+ *     (This is the normal method to access it.)
+ *
+ *     $ NET USE Z: \\server\share
+ *     $ git -C Z:/repo status
+ *
+ * (b) Windows allows a network share to be referenced WITHOUT mapping
+ *     it to drive letter.
+ *
+ *     $ NET USE \\server\share\dir
+ *     $ git -C //server/share/repo status
+ *
+ * (c) Windows allows "SUBST" to create a fake drive mapping to an
+ *     arbitrary path (which may be remote)
+ *
+ *     $ SUBST Q: Z:\repo
+ *     $ git -C Q:/ status
+ *
+ * (d) Windows allows a directory symlink to be created on a local
+ *     file system that points to a remote repo.
+ *
+ *     $ mklink /d ./link //server/share/repo
+ *     $ git -C ./link status
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	wchar_t wpath[MAX_PATH];
+	wchar_t wfullpath[MAX_PATH];
+	size_t wlen;
+	UINT driveType;
+
+	/*
+	 * Do everything in wide chars because the drive letter might be
+	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
+	 */
+	if (xutftowcs_path(wpath, path) < 0) {
+		return -1;
+	}
+
+	/*
+	 * GetDriveTypeW() requires a final slash.  We assume that the
+	 * worktree pathname points to an actual directory.
+	 */
+	wlen = wcslen(wpath);
+	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
+		wpath[wlen++] = L'\\';
+		wpath[wlen] = 0;
+	}
+
+	/*
+	 * Normalize the path.  If nothing else, this converts forward
+	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
+	 * correctly handle some UNC "\\server\share\..." paths.
+	 */
+	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) {
+		return -1;
+	}
+
+	driveType = GetDriveTypeW(wfullpath);
+	trace_printf_key(&trace_fsmonitor,
+			 "DriveType '%s' L'%ls' (%u)",
+			 path, wfullpath, driveType);
+
+	if (driveType == DRIVE_REMOTE)
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dba3ced6bb7 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -1,32 +1,10 @@
-#include "cache.h"
 #include "config.h"
-#include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
-#include <sys/param.h>
-#include <sys/mount.h>
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
-/*
- * [1] Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type (NFS, SAMBA, etc.) dictates whether notification events
- * are available at all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
+ /*
  * For the builtin FSMonitor, we create the Unix domain socket for the
  * IPC in the .git directory.  If the working directory is remote,
  * then the socket will be created on the remote file system.  This
@@ -38,40 +16,34 @@
  * be taken to ensure that $HOME is actually local and not a managed
  * file share.)
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
+ * FAT32 and NTFS working directories are problematic too.
  *
  * The builtin FSMonitor uses a Unix domain socket in the .git
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
  */
-static enum fsmonitor_reason check_volume(struct repository *r)
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
-	struct statfs fs;
+	struct fs_info fs;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	struct strbuf path = STRBUF_INIT;
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
-	if (statfs(r->worktree, &fs) == -1) {
-		int saved_errno = errno;
-		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-				 r->worktree, strerror(saved_errno));
-		errno = saved_errno;
+	if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) {
+		strbuf_release(&path);
 		return FSMONITOR_REASON_ERROR;
 	}
 
-	trace_printf_key(&trace_fsmonitor,
-			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
 
-	if (!(fs.f_flags & MNT_LOCAL))
+	if (fs.is_remote)
 		return FSMONITOR_REASON_REMOTE;
 
-	if (!strcmp(fs.f_fstypename, "msdos")) /* aka FAT32 */
+	if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
 		return FSMONITOR_REASON_NOSOCKETS;
 
-	if (!strcmp(fs.f_fstypename, "ntfs"))
+	if (!strcmp(fs.typename, "ntfs"))
 		return FSMONITOR_REASON_NOSOCKETS;
 
 	return FSMONITOR_REASON_OK;
@@ -81,7 +53,7 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_volume(r);
+	reason = check_uds_volume(r);
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index 907655720bb..d88b06ae610 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -1,8 +1,9 @@
 #include "cache.h"
 #include "config.h"
 #include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * VFS for Git is incompatible with FSMonitor.
@@ -24,103 +25,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type dictates whether notification events are available at
- * all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- * Notes for testing:
- *
- * (a) Windows allows a network share to be mapped to a drive letter.
- *     (This is the normal method to access it.)
- *
- *     $ NET USE Z: \\server\share
- *     $ git -C Z:/repo status
- *
- * (b) Windows allows a network share to be referenced WITHOUT mapping
- *     it to drive letter.
- *
- *     $ NET USE \\server\share\dir
- *     $ git -C //server/share/repo status
- *
- * (c) Windows allows "SUBST" to create a fake drive mapping to an
- *     arbitrary path (which may be remote)
- *
- *     $ SUBST Q: Z:\repo
- *     $ git -C Q:/ status
- *
- * (d) Windows allows a directory symlink to be created on a local
- *     file system that points to a remote repo.
- *
- *     $ mklink /d ./link //server/share/repo
- *     $ git -C ./link status
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
-	wchar_t wpath[MAX_PATH];
-	wchar_t wfullpath[MAX_PATH];
-	size_t wlen;
-	UINT driveType;
-
-	/*
-	 * Do everything in wide chars because the drive letter might be
-	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
-	 */
-	if (xutftowcs_path(wpath, r->worktree) < 0)
-		return FSMONITOR_REASON_ERROR;
-
-	/*
-	 * GetDriveTypeW() requires a final slash.  We assume that the
-	 * worktree pathname points to an actual directory.
-	 */
-	wlen = wcslen(wpath);
-	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
-		wpath[wlen++] = L'\\';
-		wpath[wlen] = 0;
-	}
-
-	/*
-	 * Normalize the path.  If nothing else, this converts forward
-	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
-	 * correctly handle some UNC "\\server\share\..." paths.
-	 */
-	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
-		return FSMONITOR_REASON_ERROR;
-
-	driveType = GetDriveTypeW(wfullpath);
-	trace_printf_key(&trace_fsmonitor,
-			 "DriveType '%s' L'%ls' (%u)",
-			 r->worktree, wfullpath, driveType);
-
-	if (driveType == DRIVE_REMOTE) {
-		trace_printf_key(&trace_fsmonitor,
-				 "check_remote('%s') true",
-				 r->worktree);
-		return FSMONITOR_REASON_REMOTE;
-	}
-
-	return FSMONITOR_REASON_OK;
-}
-
 enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
@@ -129,9 +33,5 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
-	reason = check_remote(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
-
 	return FSMONITOR_REASON_OK;
 }
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..b88494bf59b 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
new file mode 100644
index 00000000000..e48592887e7
--- /dev/null
+++ b/fsmonitor-path-utils.h
@@ -0,0 +1,23 @@
+#ifndef FSM_PATH_UTILS_H
+#define FSM_PATH_UTILS_H
+
+struct fs_info {
+	int is_remote;
+	char *typename;
+};
+
+/*
+ * Get some basic filesystem informtion for the given path
+ *
+ * Returns -1 on error, zero otherwise.
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+
+/*
+ * Determines if the filesystem that path resides on is remote.
+ *
+ * Returns -1 on error, 0 if not remote, 1 if remote.
+ */
+int fsmonitor__is_fs_remote(const char *path);
+
+#endif
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..d288cbad479 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v5 2/4] fsmonitor: relocate socket file if .git directory is remote
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 1/4] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
@ 2022-09-10 20:00         ` Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 3/4] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
                           ` (4 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-10 20:00 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If the .git directory is on a remote file system, create the socket
file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                            |  1 +
 compat/fsmonitor/fsm-ipc-darwin.c   | 46 +++++++++++++++++++++++++++++
 compat/fsmonitor/fsm-ipc-win32.c    |  4 +++
 contrib/buildsystems/CMakeLists.txt |  2 ++
 fsmonitor-ipc.c                     |  2 --
 5 files changed, 53 insertions(+), 2 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c

diff --git a/Makefile b/Makefile
index 265fc585286..cf9c51040a0 100644
--- a/Makefile
+++ b/Makefile
@@ -2037,6 +2037,7 @@ ifdef FSMONITOR_DAEMON_BACKEND
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
 endif
 
 ifdef FSMONITOR_OS_SETTINGS
diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
new file mode 100644
index 00000000000..afaca96dab9
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-darwin.c
@@ -0,0 +1,46 @@
+#include "cache.h"
+#include "config.h"
+#include "strbuf.h"
+#include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
+	static const char *ipc_path;
+	SHA_CTX sha1ctx;
+	char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (ipc_path)
+		return ipc_path;
+
+	ipc_path = fsmonitor_ipc__get_default_path();
+
+	/* By default the socket file is created in the .git directory */
+	if (fsmonitor__is_fs_remote(ipc_path) < 1)
+		return ipc_path;
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
+	SHA1_Final(hash, &sha1ctx);
+
+	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
+
+	/* Create the socket file in either socketDir or $HOME */
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+
+	ipc_path = interpolate_path(ipc_file.buf, 1);
+	if (!ipc_path)
+		die(_("Invalid path: %s"), ipc_file.buf);
+
+	strbuf_release(&ipc_file);
+	return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
new file mode 100644
index 00000000000..769a88639f6
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-win32.c
@@ -0,0 +1,4 @@
+#include "cache.h"
+#include "fsmonitor-ipc.h"
+
+GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index b88494bf59b..7e7b6b9a362 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..caad2e246a0 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
 	return ipc_get_active_state(fsmonitor_ipc__get_path());
-- 
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
 	return ipc_get_active_state(fsmonitor_ipc__get_path());
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v5 3/4] fsmonitor: avoid socket location check if using hook
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 1/4] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 2/4] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-10 20:00         ` Eric DeCosta via GitGitGadget
  2022-09-10 20:00         ` [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
                           ` (3 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-10 20:00 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If monitoring is done via fsmonitor hook rather than IPC there is no
need to check if the location of the Unix Domain socket (UDS) file is
on a remote filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
 compat/fsmonitor/fsm-settings-win32.c  |  2 +-
 fsmonitor-settings.c                   |  8 ++++----
 fsmonitor-settings.h                   |  2 +-
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index dba3ced6bb7..3463c71763e 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_uds_volume(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
+	if (ipc) {
+		reason = check_uds_volume(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
+	}
 
 	return FSMONITOR_REASON_OK;
 }
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index d88b06ae610..a8af31b71de 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index d288cbad479..531a1b6f956 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 }
 #endif
 
-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
 {
 	if (!r->worktree) {
 		/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 		reason = check_remote(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-		reason = fsm_os__incompatible(r);
+		reason = fsm_os__incompatible(r, ipc);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
 	}
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..0721617b95a 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                           ` (2 preceding siblings ...)
  2022-09-10 20:00         ` [PATCH v5 3/4] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-10 20:00         ` Eric DeCosta via GitGitGadget
  2022-09-11  1:01           ` Eric Sunshine
  2022-09-12 15:27         ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
                           ` (2 subsequent siblings)
  6 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-10 20:00 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path invloves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     |  6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 91 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  6 ++
 fsmonitor-path-utils.h                   | 26 +++++++
 6 files changed, 153 insertions(+), 1 deletion(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 2c109cf8b37..2559150d9e7 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	strbuf_init(&state.alias.alias, 0);
+	strbuf_init(&state.alias.points_to, 0);
+	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+		err = error(_("could not get worktree alias"));
+		goto done;
+	}
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..179886bc15b 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (!path_k)
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index 067cbe6990a..1f1550c485a 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -38,3 +41,91 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogteher.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR * dir;
+	int read;
+	int retval;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to;
+
+	retval = 0;
+	dir = opendir("/");
+	if (!dir)
+		return -1;
+
+	strbuf_init(&alias, 256);
+	strbuf_init(&points_to, MAXPATHLEN);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addch(&alias, '/');
+		strbuf_add(&alias, de->d_name, strlen(de->d_name));
+
+		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
+		if (read > 0) {
+			strbuf_setlen(&points_to, read);
+			if ((strncmp(points_to.buf, path, points_to.len) == 0)
+				&& path[points_to.len] == '/') {
+				strbuf_addbuf(&info->alias, &alias);
+				strbuf_addbuf(&info->points_to, &points_to);
+				trace_printf_key(&trace_fsmonitor,
+					"Found alias for '%s' : '%s' -> '%s'",
+					path, info->alias.buf, info->points_to.buf);
+				retval = 0;
+				goto done;
+			}
+		} else if (errno != EINVAL) { /* Something other than not a link */
+			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
+			retval = -1;
+			goto done;
+		}
+	}
+
+	done:
+	closedir(dir);
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	int len = info->alias.len;
+
+	if (!len)
+		return NULL;
+
+	if ((strncmp(info->alias.buf, path, len) == 0)
+		&& path[len] == '/') {
+		struct strbuf tmp;
+		const char *remainder = path + len;
+		int rem_len = strlen(remainder);
+
+		strbuf_init(&tmp, info->points_to.len + rem_len);
+		strbuf_addbuf(&tmp, &info->points_to);
+		strbuf_add(&tmp, remainder, rem_len);
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index dc211de514f..0ca366bfe08 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -87,3 +87,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..98cbb430083 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
@@ -167,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,
  */
 void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
 
+char *fsmonitor_resolve_alias(const char *path,
+	struct alias_info *alias);
+
 #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
 #endif /* FSMONITOR_DAEMON_H */
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index e48592887e7..7d9d71cb043 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	struct strbuf alias;
+	struct strbuf points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,22 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * Returns -1 on error, 0 otherwise.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	struct strbuf alias;
+	struct strbuf points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,22 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * Returns -1 on error, 0 otherwise.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-10 20:00         ` [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-11  1:01           ` Eric Sunshine
  0 siblings, 0 replies; 157+ messages in thread
From: Eric Sunshine @ 2022-09-11  1:01 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

On Sat, Sep 10, 2022 at 4:00 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Starting with macOS 10.15 (Catalina), Apple introduced a new feature
> called 'firmlinks' in order to separate the boot volume into two
> volumes, one read-only and one writable but still present them to the
> user as a single volume. Along with this change, Apple removed the
> ability to create symlinks in the root directory and replaced them with
> 'synthetic firmlinks'. See 'man synthetic.conf'
>
> When FSEevents reports the path of changed files, if the path invloves
> a synthetic firmlink, the path is reported from the point of the
> synthetic firmlink and not the real path. For example:

s/invloves/involves/

> Real path:
> /System/Volumes/Data/network/working/directory/foo.txt
>
> Synthetic firmlink:
> /network -> /System/Volumes/Data/network
>
> FSEvents path:
> /network/working/directory/foo.txt
>
> This causes the FSEvents path to not match against the worktree
> directory.
>
> There are several ways in which synthetic firmlinks can be created:
> they can be defined in /etc/synthetic.conf, the automounter can create
> them, and there may be other means. Simply reading /etc/synthetic.conf
> is insufficient. No matter what process creates synthetic firmlinks,
> they all get created in the root directory.
>
> Therefore, in order to deal with synthetic firmlinks, the root directory
> is scanned and the first possible synthetic firmink that, when resolved,
> is a prefix of the worktree is used to map FSEvents paths to worktree
> paths.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
> diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
> @@ -38,3 +41,91 @@ int fsmonitor__is_fs_remote(const char *path)
> +/*
> + * Scan the root directory for synthetic firmlinks that when resolved
> + * are a prefix of the path, stopping at the first one found.
> + *
> + * Some information about firmlinks and synthetic firmlinks:
> + * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
> + *
> + * macOS no longer allows symlinks in the root directory; any link found
> + * there is therefore a synthetic firmlink.
> + *
> + * If this function gets called often, will want to cache all the firmlink
> + * information, but for now there is only one caller of this function.
> + *
> + * If there is more than one alias for the path, that is another
> + * matter altogteher.
> + */

s/altogteher/altogether/

> diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
> @@ -1,6 +1,14 @@
> +struct alias_info
> +{
> +       struct strbuf alias;
> +       struct strbuf points_to;
> +};
> +/*
> + * Get the alias in given path, if any.
> + *
> + * Sets alias to the first alias that matches any part of the path.
> + *
> + * Returns -1 on error, 0 otherwise.
> + */
> +int fsmonitor__get_alias(const char *path, struct alias_info *info);

I suppose it's somewhat clear here that the caller is responsible for
releasing the strbufs in alias_info (though it's not clear why they
need to be strbufs in the first place)...

> +/*
> + * Resolve the path against the given alias.
> + *
> + * Returns the resolved path if there is one, NULL otherwise.
> + */
> +char *fsmonitor__resolve_alias(const char *path,
> +       const struct alias_info *info);

But what about the return value from this function? It's `char*` which
suggests a possible ownership transfer(?). Is the caller responsible
for freeing it? If so, perhaps the documentation could state that.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                           ` (3 preceding siblings ...)
  2022-09-10 20:00         ` [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-12 15:27         ` Junio C Hamano
  2022-09-12 19:37           ` Junio C Hamano
  2022-09-12 15:35         ` Junio C Hamano
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
  6 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-12 15:27 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Follow-on to the work done to allow Windows to work against network-mounted
> repos for macOS.
>
> Have macOS take advantage of the same configuration option,
> 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
> to true will override the default behavior (erroring-out) when a
> network-mounted repo is detected by fsmonitor.
>
> The added wrinkle being that the Unix domain socket (UDS) file used for IPC
> cannot be created in a network location; instead $HOME is used if the
> default location is on the network. The user may, optionally, set the
> 'fsmonitor.socketDir' configuration option to a valid, local directory if
> $HOME itself is on the network or is simply not the desired location for the
> UDS file.
>
> An additional issue is that for mount points in the root directory, FSEvents
> does not report a path that matches the worktree directory due to the
> introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
> to the worktree directory by interrogating the root filesystem for synthetic
> firmlinks and using that information to translate the path.
>
> Eric DeCosta (4):
>   fsmonitor: refactor filesystem checks to common interface
>   fsmonitor: relocate socket file if .git directory is remote
>   fsmonitor: avoid socket location check if using hook
>   fsmonitor: deal with synthetic firmlinks on macOS

It looks like this one is organized quite differently from the last
iteration <pull.1326.v4.git.1661962145.gitgitgadget@gmail.com>

Unless the changes since the last iteration is so obvious [*] that
range-diff will explain all of it, it would help the topic to make
it easier for reviewers what has changed relative to the previous
one in your own words here in the cover letter.

[*} Here, "obvious" does not mean "diff is small and can be read in
10 minutes".  It is "diff shows the reason why these changes are
made, so there is no need for additional explanation", e.g. typofixes,
renaming of variables, making a helper function that was added as
extern in the previous iteration to static, etc.

Thanks.

>  Makefile                                 |   2 +
>  builtin/fsmonitor--daemon.c              |   8 ++
>  compat/fsmonitor/fsm-ipc-darwin.c        |  46 ++++++++
>  compat/fsmonitor/fsm-ipc-win32.c         |   4 +
>  compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
>  compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++
>  compat/fsmonitor/fsm-path-utils-win32.c  | 106 ++++++++++++++++++
>  compat/fsmonitor/fsm-settings-darwin.c   |  70 ++++--------
>  compat/fsmonitor/fsm-settings-win32.c    | 106 +-----------------
>  contrib/buildsystems/CMakeLists.txt      |   4 +
>  fsmonitor--daemon.h                      |   6 ++
>  fsmonitor-ipc.c                          |   2 -
>  fsmonitor-path-utils.h                   |  49 +++++++++
>  fsmonitor-settings.c                     |  58 +++++++++-
>  fsmonitor-settings.h                     |   2 +-
>  15 files changed, 441 insertions(+), 159 deletions(-)
>  create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
>  create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
>  create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
>  create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
>  create mode 100644 fsmonitor-path-utils.h
>
>
> base-commit: be1a02a17ede4082a86dfbfee0f54f345e8b43ac
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v5
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v5
> Pull-Request: https://github.com/gitgitgadget/git/pull/1326
>
> Range-diff vs v4:
>
>  -:  ----------- > 1:  2f647b53e4d fsmonitor: refactor filesystem checks to common interface
>  2:  2cb026a6317 ! 2:  28d08bcf808 fsmonitor: generate unique Unix socket file name in the desired location
>      @@ Metadata
>       Author: Eric DeCosta <edecosta@mathworks.com>
>       
>        ## Commit message ##
>      -    fsmonitor: generate unique Unix socket file name in the desired location
>      +    fsmonitor: relocate socket file if .git directory is remote
>       
>      -    Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir
>      -    locate the Unix domain socket file in the desired location (either
>      -    the .git directory, $HOME, or fsmonitor.socketDir). If the location
>      -    is other than the .git directory, generate a unique file name based
>      -    on the SHA1 has of the path to the .git directory.
>      +    If the .git directory is on a remote file system, create the socket
>      +    file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
>       
>           Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
>       
>      - ## fsmonitor-ipc.c ##
>      -@@
>      - #include "cache.h"
>      --#include "fsmonitor.h"
>      --#include "simple-ipc.h"
>      - #include "fsmonitor-ipc.h"
>      -+#include "fsmonitor-settings.h"
>      - #include "run-command.h"
>      - #include "strbuf.h"
>      - #include "trace2.h"
>      -@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
>      - 	return 1;
>      - }
>      + ## Makefile ##
>      +@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND
>      + 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
>      + 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
>      + 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
>      ++	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
>      + endif
>        
>      --GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
>      -+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
>      + ifdef FSMONITOR_OS_SETTINGS
>      +
>      + ## compat/fsmonitor/fsm-ipc-darwin.c (new) ##
>      +@@
>      ++#include "cache.h"
>      ++#include "config.h"
>      ++#include "strbuf.h"
>      ++#include "fsmonitor.h"
>      ++#include "fsmonitor-ipc.h"
>      ++#include "fsmonitor-path-utils.h"
>      ++
>      ++static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
>       +
>       +const char *fsmonitor_ipc__get_path(void)
>       +{
>      -+#ifdef WIN32
>      -+	return fsmonitor_ipc__get_default_path();
>      -+#else
>      -+	char *retval;
>      ++	static const char *ipc_path;
>       +	SHA_CTX sha1ctx;
>      -+	const char *git_dir;
>      -+	const char *sock_dir;
>      ++	char *sock_dir;
>       +	struct strbuf ipc_file = STRBUF_INIT;
>       +	unsigned char hash[SHA_DIGEST_LENGTH];
>       +
>      -+	if (fsm_settings__get_allow_remote(the_repository) < 1)
>      -+		return fsmonitor_ipc__get_default_path();
>      ++	if (ipc_path)
>      ++		return ipc_path;
>       +
>      -+	git_dir = get_git_dir();
>      -+	sock_dir = fsm_settings__get_socket_dir(the_repository);
>      ++	ipc_path = fsmonitor_ipc__get_default_path();
>      ++
>      ++	/* By default the socket file is created in the .git directory */
>      ++	if (fsmonitor__is_fs_remote(ipc_path) < 1)
>      ++		return ipc_path;
>       +
>       +	SHA1_Init(&sha1ctx);
>      -+	SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
>      ++	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
>       +	SHA1_Final(hash, &sha1ctx);
>       +
>      ++	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
>      ++
>      ++	/* Create the socket file in either socketDir or $HOME */
>       +	if (sock_dir && *sock_dir)
>       +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
>       +					sock_dir, hash_to_hex(hash));
>       +	else
>       +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
>      -+	retval = interpolate_path(ipc_file.buf, 1);
>      -+	if (!retval)
>      ++
>      ++	ipc_path = interpolate_path(ipc_file.buf, 1);
>      ++	if (!ipc_path)
>       +		die(_("Invalid path: %s"), ipc_file.buf);
>      ++
>       +	strbuf_release(&ipc_file);
>      -+	return retval;
>      -+#endif
>      ++	return ipc_path;
>       +}
>      +
>      + ## compat/fsmonitor/fsm-ipc-win32.c (new) ##
>      +@@
>      ++#include "cache.h"
>      ++#include "fsmonitor-ipc.h"
>      ++
>      ++GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
>      +
>      + ## contrib/buildsystems/CMakeLists.txt ##
>      +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
>      + 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
>      ++		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
>        
>      - enum ipc_active_state fsmonitor_ipc__get_state(void)
>      - {
>      + 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
>      +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
>      + 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
>      ++		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
>      + 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
>      + 
>      + 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
>       
>      - ## fsmonitor-ipc.h ##
>      -@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
>      -  */
>      - const char *fsmonitor_ipc__get_path(void);
>      + ## fsmonitor-ipc.c ##
>      +@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
>      + 	return 1;
>      + }
>        
>      -+/*
>      -+ * Returns the pathname to the default IPC named pipe or Unix domain
>      -+ * socket.
>      -+ */
>      -+const char *fsmonitor_ipc__get_default_path(void);
>      -+
>      - /*
>      -  * Try to determine whether there is a `git-fsmonitor--daemon` process
>      -  * listening on the IPC pipe/socket.
>      +-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
>      +-
>      + enum ipc_active_state fsmonitor_ipc__get_state(void)
>      + {
>      + 	return ipc_get_active_state(fsmonitor_ipc__get_path());
>  1:  836a791e6b7 ! 3:  ff54b6e0bb5 fsmonitor: add two new config options, allowRemote and socketDir
>      @@ Metadata
>       Author: Eric DeCosta <edecosta@mathworks.com>
>       
>        ## Commit message ##
>      -    fsmonitor: add two new config options, allowRemote and socketDir
>      +    fsmonitor: avoid socket location check if using hook
>       
>      -    Introduce two new configuration options
>      -
>      -       fsmonitor.allowRemote - setting this to true overrides fsmonitor's
>      -       default behavior of erroring out when enountering network file
>      -       systems. Additionly, when true, the Unix domain socket (UDS) file
>      -       used for IPC is located in $HOME rather than in the .git directory.
>      -
>      -       fsmonitor.socketDir - allows for the UDS file to be located
>      -       anywhere the user chooses rather $HOME.
>      +    If monitoring is done via fsmonitor hook rather than IPC there is no
>      +    need to check if the location of the Unix Domain socket (UDS) file is
>      +    on a remote filesystem.
>       
>           Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
>       
>      - ## fsmonitor-settings.c ##
>      -@@
>      - struct fsmonitor_settings {
>      - 	enum fsmonitor_mode mode;
>      + ## compat/fsmonitor/fsm-settings-darwin.c ##
>      +@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_uds_volume(struct repository *r)
>      + 	return FSMONITOR_REASON_OK;
>      + }
>      + 
>      +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
>      ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
>      + {
>        	enum fsmonitor_reason reason;
>      -+	int allow_remote;
>      - 	char *hook_path;
>      -+	char *sock_dir;
>      - };
>        
>      - static enum fsmonitor_reason check_for_incompatible(struct repository *r)
>      -@@ fsmonitor-settings.c: static struct fsmonitor_settings *alloc_settings(void)
>      - 	CALLOC_ARRAY(s, 1);
>      - 	s->mode = FSMONITOR_MODE_DISABLED;
>      - 	s->reason = FSMONITOR_REASON_UNTESTED;
>      -+	s->allow_remote = -1;
>      +-	reason = check_uds_volume(r);
>      +-	if (reason != FSMONITOR_REASON_OK)
>      +-		return reason;
>      ++	if (ipc) {
>      ++		reason = check_uds_volume(r);
>      ++		if (reason != FSMONITOR_REASON_OK)
>      ++			return reason;
>      ++	}
>        
>      - 	return s;
>      + 	return FSMONITOR_REASON_OK;
>        }
>      -@@ fsmonitor-settings.c: static void lookup_fsmonitor_settings(struct repository *r)
>      - 		fsm_settings__set_disabled(r);
>      +
>      + ## compat/fsmonitor/fsm-settings-win32.c ##
>      +@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4git(struct repository *r)
>      + 	return FSMONITOR_REASON_OK;
>        }
>        
>      -+int fsm_settings__get_allow_remote(struct repository *r)
>      -+{
>      -+	if (!r)
>      -+		r = the_repository;
>      -+	if (!r->settings.fsmonitor)
>      -+		lookup_fsmonitor_settings(r);
>      -+
>      -+	return r->settings.fsmonitor->allow_remote;
>      -+}
>      -+
>      -+const char *fsm_settings__get_socket_dir(struct repository *r)
>      -+{
>      -+	if (!r)
>      -+		r = the_repository;
>      -+	if (!r->settings.fsmonitor)
>      -+		lookup_fsmonitor_settings(r);
>      -+
>      -+	return r->settings.fsmonitor->sock_dir;
>      -+}
>      -+
>      - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>      +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
>      ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
>        {
>      - 	if (!r)
>      -@@ fsmonitor-settings.c: enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
>      - 	return r->settings.fsmonitor->mode;
>      + 	enum fsmonitor_reason reason;
>      + 
>      +
>      + ## fsmonitor-settings.c ##
>      +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_remote(struct repository *r)
>        }
>      + #endif
>        
>      -+
>      - const char *fsm_settings__get_hook_path(struct repository *r)
>      +-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
>      ++static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
>        {
>      - 	if (!r)
>      + 	if (!r->worktree) {
>      + 		/*
>      +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_for_incompatible(struct repository *r)
>      + 		reason = check_remote(r);
>      + 		if (reason != FSMONITOR_REASON_OK)
>      + 			return reason;
>      +-		reason = fsm_os__incompatible(r);
>      ++		reason = fsm_os__incompatible(r, ipc);
>      + 		if (reason != FSMONITOR_REASON_OK)
>      + 			return reason;
>      + 	}
>       @@ fsmonitor-settings.c: const char *fsm_settings__get_hook_path(struct repository *r)
>      - 	return r->settings.fsmonitor->hook_path;
>      - }
>        
>      -+void fsm_settings__set_allow_remote(struct repository *r)
>      -+{
>      -+	int allow;
>      -+
>      -+	if (!r)
>      -+		r = the_repository;
>      -+	if (!r->settings.fsmonitor)
>      -+		r->settings.fsmonitor = alloc_settings();
>      -+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
>      -+		r->settings.fsmonitor->allow_remote = allow;
>      -+
>      -+	return;
>      -+}
>      -+
>      -+void fsm_settings__set_socket_dir(struct repository *r)
>      -+{
>      -+	const char *path;
>      -+
>      -+	if (!r)
>      -+		r = the_repository;
>      -+	if (!r->settings.fsmonitor)
>      -+		r->settings.fsmonitor = alloc_settings();
>      -+
>      -+	if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
>      -+		FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
>      -+		r->settings.fsmonitor->sock_dir = strdup(path);
>      -+	}
>      -+
>      -+	return;
>      -+}
>      -+
>        void fsm_settings__set_ipc(struct repository *r)
>        {
>       -	enum fsmonitor_reason reason = check_for_incompatible(r);
>      -+	enum fsmonitor_reason reason;
>      -+
>      -+	fsm_settings__set_allow_remote(r);
>      -+	fsm_settings__set_socket_dir(r);
>      -+	reason = check_for_incompatible(r);
>      ++	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
>        
>        	if (reason != FSMONITOR_REASON_OK) {
>        		fsm_settings__set_incompatible(r, reason);
>      @@ fsmonitor-settings.c: void fsm_settings__set_ipc(struct repository *r)
>        void fsm_settings__set_hook(struct repository *r, const char *path)
>        {
>       -	enum fsmonitor_reason reason = check_for_incompatible(r);
>      -+	enum fsmonitor_reason reason;
>      -+
>      -+	fsm_settings__set_allow_remote(r);
>      -+	fsm_settings__set_socket_dir(r);
>      -+	reason = check_for_incompatible(r);
>      ++	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
>        
>        	if (reason != FSMONITOR_REASON_OK) {
>        		fsm_settings__set_incompatible(r, reason);
>       
>        ## fsmonitor-settings.h ##
>      -@@ fsmonitor-settings.h: enum fsmonitor_reason {
>      - 	FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
>      - };
>      - 
>      -+void fsm_settings__set_allow_remote(struct repository *r);
>      -+void fsm_settings__set_socket_dir(struct repository *r);
>      - void fsm_settings__set_ipc(struct repository *r);
>      - void fsm_settings__set_hook(struct repository *r, const char *path);
>      - void fsm_settings__set_disabled(struct repository *r);
>      - void fsm_settings__set_incompatible(struct repository *r,
>      - 				    enum fsmonitor_reason reason);
>      - 
>      -+int fsm_settings__get_allow_remote(struct repository *r);
>      -+const char *fsm_settings__get_socket_dir(struct repository *r);
>      - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
>      - const char *fsm_settings__get_hook_path(struct repository *r);
>      +@@ fsmonitor-settings.h: struct fsmonitor_settings;
>      +  * fsm_os__* routines should considered private to fsm_settings__
>      +  * routines.
>      +  */
>      +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
>      ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
>      + #endif /* HAVE_FSMONITOR_OS_SETTINGS */
>        
>      + #endif /* FSMONITOR_SETTINGS_H */
>  3:  a3110f1e25a < -:  ----------- fsmonitor: ensure filesystem and unix socket filesystem are compatible
>  4:  56cabf3be3b ! 4:  b7d6cf44695 fsmonitor: normalize FSEvents event paths to the real path
>      @@ Metadata
>       Author: Eric DeCosta <edecosta@mathworks.com>
>       
>        ## Commit message ##
>      -    fsmonitor: normalize FSEvents event paths to the real path
>      +    fsmonitor: deal with synthetic firmlinks on macOS
>       
>      -    Consider the following network working directory that is mounted under
>      -    /System/Volumes/Data:
>      +    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
>      +    called 'firmlinks' in order to separate the boot volume into two
>      +    volumes, one read-only and one writable but still present them to the
>      +    user as a single volume. Along with this change, Apple removed the
>      +    ability to create symlinks in the root directory and replaced them with
>      +    'synthetic firmlinks'. See 'man synthetic.conf'
>       
>      -    /network/working/directory
>      +    When FSEevents reports the path of changed files, if the path invloves
>      +    a synthetic firmlink, the path is reported from the point of the
>      +    synthetic firmlink and not the real path. For example:
>       
>      -    The git working directory path is:
>      +    Real path:
>      +    /System/Volumes/Data/network/working/directory/foo.txt
>       
>      -    /System/Volumes/Data/network/working/directory
>      +    Synthetic firmlink:
>      +    /network -> /System/Volumes/Data/network
>       
>      -    The paths reported by FSEvents always start with /network. fsmonitor
>      -    expects paths to be under the working directory; therefore it
>      -    fails to match /network/... and ignores the change.
>      +    FSEvents path:
>      +    /network/working/directory/foo.txt
>       
>      -    Change things such that if fsmonitor.allowRemote is true that the
>      -    paths reported via FSEevents are normalized to the real path.
>      +    This causes the FSEvents path to not match against the worktree
>      +    directory.
>      +
>      +    There are several ways in which synthetic firmlinks can be created:
>      +    they can be defined in /etc/synthetic.conf, the automounter can create
>      +    them, and there may be other means. Simply reading /etc/synthetic.conf
>      +    is insufficient. No matter what process creates synthetic firmlinks,
>      +    they all get created in the root directory.
>      +
>      +    Therefore, in order to deal with synthetic firmlinks, the root directory
>      +    is scanned and the first possible synthetic firmink that, when resolved,
>      +    is a prefix of the worktree is used to map FSEvents paths to worktree
>      +    paths.
>       
>           Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
>       
>      + ## builtin/fsmonitor--daemon.c ##
>      +@@
>      + #include "parse-options.h"
>      + #include "fsmonitor.h"
>      + #include "fsmonitor-ipc.h"
>      ++#include "fsmonitor-path-utils.h"
>      + #include "compat/fsmonitor/fsm-health.h"
>      + #include "compat/fsmonitor/fsm-listen.h"
>      + #include "fsmonitor--daemon.h"
>      +@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
>      + 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
>      + 	state.nr_paths_watching = 1;
>      + 
>      ++	strbuf_init(&state.alias.alias, 0);
>      ++	strbuf_init(&state.alias.points_to, 0);
>      ++	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
>      ++		err = error(_("could not get worktree alias"));
>      ++		goto done;
>      ++	}
>      ++
>      + 	/*
>      + 	 * We create and delete cookie files somewhere inside the .git
>      + 	 * directory to help us keep sync with the file system.  If
>      +
>        ## compat/fsmonitor/fsm-listen-darwin.c ##
>       @@
>        #include "fsmonitor.h"
>        #include "fsm-listen.h"
>        #include "fsmonitor--daemon.h"
>      -+#include "fsmonitor-settings.h"
>      ++#include "fsmonitor-path-utils.h"
>        
>        struct fsm_listen_data
>        {
>      -@@ compat/fsmonitor/fsm-listen-darwin.c: static void my_add_path(struct fsmonitor_batch *batch, const char *path)
>      - 	free(composed);
>      - }
>      - 
>      --
>      - static void fsevent_callback(ConstFSEventStreamRef streamRef,
>      - 			     void *ctx,
>      - 			     size_t num_of_events,
>       @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
>        		/*
>        		 * On Mac, we receive an array of absolute paths.
>        		 */
>       -		path_k = paths[k];
>      -+		if (fsm_settings__get_allow_remote(the_repository) > 0) {
>      -+			strbuf_reset(&tmp);
>      -+			strbuf_realpath_forgiving(&tmp, paths[k], 0);
>      -+			path_k = tmp.buf;
>      -+		} else
>      ++		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
>      ++		if (!path_k)
>       +			path_k = paths[k];
>        
>        		/*
>        		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
>       @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
>      - 
>        			fsmonitor_force_resync(state);
>        			fsmonitor_batch__free_list(batch);
>      -+			batch = NULL;
>        			string_list_clear(&cookie_list, 0);
>      ++			batch = NULL;
>        
>        			/*
>      -@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
>      + 			 * We assume that any events that we received
>      +
>      + ## compat/fsmonitor/fsm-path-utils-darwin.c ##
>      +@@
>      + #include "fsmonitor.h"
>      + #include "fsmonitor-path-utils.h"
>      ++#include <dirent.h>
>      ++#include <errno.h>
>      ++#include <fcntl.h>
>      + #include <sys/param.h>
>      + #include <sys/mount.h>
>      + 
>      +@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char *path)
>      + 		return -1;
>      + 	return fs.is_remote;
>      + }
>      ++
>      ++/*
>      ++ * Scan the root directory for synthetic firmlinks that when resolved
>      ++ * are a prefix of the path, stopping at the first one found.
>      ++ *
>      ++ * Some information about firmlinks and synthetic firmlinks:
>      ++ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
>      ++ *
>      ++ * macOS no longer allows symlinks in the root directory; any link found
>      ++ * there is therefore a synthetic firmlink.
>      ++ *
>      ++ * If this function gets called often, will want to cache all the firmlink
>      ++ * information, but for now there is only one caller of this function.
>      ++ *
>      ++ * If there is more than one alias for the path, that is another
>      ++ * matter altogteher.
>      ++ */
>      ++int fsmonitor__get_alias(const char *path, struct alias_info *info)
>      ++{
>      ++	DIR * dir;
>      ++	int read;
>      ++	int retval;
>      ++	struct dirent *de;
>      ++	struct strbuf alias;
>      ++	struct strbuf points_to;
>      ++
>      ++	retval = 0;
>      ++	dir = opendir("/");
>      ++	if (!dir)
>      ++		return -1;
>      ++
>      ++	strbuf_init(&alias, 256);
>      ++	strbuf_init(&points_to, MAXPATHLEN);
>      ++
>      ++	while ((de = readdir(dir)) != NULL) {
>      ++		strbuf_reset(&alias);
>      ++		strbuf_addch(&alias, '/');
>      ++		strbuf_add(&alias, de->d_name, strlen(de->d_name));
>      ++
>      ++		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
>      ++		if (read > 0) {
>      ++			strbuf_setlen(&points_to, read);
>      ++			if ((strncmp(points_to.buf, path, points_to.len) == 0)
>      ++				&& path[points_to.len] == '/') {
>      ++				strbuf_addbuf(&info->alias, &alias);
>      ++				strbuf_addbuf(&info->points_to, &points_to);
>      ++				trace_printf_key(&trace_fsmonitor,
>      ++					"Found alias for '%s' : '%s' -> '%s'",
>      ++					path, info->alias.buf, info->points_to.buf);
>      ++				retval = 0;
>      ++				goto done;
>      ++			}
>      ++		} else if (errno != EINVAL) { /* Something other than not a link */
>      ++			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
>      ++			retval = -1;
>      ++			goto done;
>      ++		}
>      ++	}
>      ++
>      ++	done:
>      ++	closedir(dir);
>      ++	strbuf_release(&alias);
>      ++	strbuf_release(&points_to);
>      ++	return retval;
>      ++}
>      ++
>      ++char *fsmonitor__resolve_alias(const char *path,
>      ++	const struct alias_info *info)
>      ++{
>      ++	int len = info->alias.len;
>      ++
>      ++	if (!len)
>      ++		return NULL;
>      ++
>      ++	if ((strncmp(info->alias.buf, path, len) == 0)
>      ++		&& path[len] == '/') {
>      ++		struct strbuf tmp;
>      ++		const char *remainder = path + len;
>      ++		int rem_len = strlen(remainder);
>      ++
>      ++		strbuf_init(&tmp, info->points_to.len + rem_len);
>      ++		strbuf_addbuf(&tmp, &info->points_to);
>      ++		strbuf_add(&tmp, remainder, rem_len);
>      ++		return strbuf_detach(&tmp, NULL);
>      ++	}
>      ++
>      ++	return NULL;
>      ++}
>      +
>      + ## compat/fsmonitor/fsm-path-utils-win32.c ##
>      +@@ compat/fsmonitor/fsm-path-utils-win32.c: int fsmonitor__is_fs_remote(const char *path)
>      + 		return -1;
>      + 	return fs.is_remote;
>      + }
>      ++
>      ++/*
>      ++ * No-op for now.
>      ++ */
>      ++int fsmonitor__get_alias(const char *path, struct alias_info *info)
>      ++{
>      ++	return 0;
>      ++}
>      ++
>      ++/*
>      ++ * No-op for now.
>      ++ */
>      ++char *fsmonitor__resolve_alias(const char *path,
>      ++	const struct alias_info *info)
>      ++{
>      ++	return NULL;
>      ++}
>      +
>      + ## fsmonitor--daemon.h ##
>      +@@
>      + #include "run-command.h"
>      + #include "simple-ipc.h"
>      + #include "thread-utils.h"
>      ++#include "fsmonitor-path-utils.h"
>      + 
>      + struct fsmonitor_batch;
>      + struct fsmonitor_token_data;
>      +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
>      + 
>      + 	struct strbuf path_worktree_watch;
>      + 	struct strbuf path_gitdir_watch;
>      ++	struct alias_info alias;
>      + 	int nr_paths_watching;
>      + 
>      + 	struct fsmonitor_token_data *current_token_data;
>      +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
>      + 
>      + 	struct ipc_server_data *ipc_server_data;
>      + 	struct strbuf path_ipc;
>      ++
>      + };
>      + 
>      + /*
>      +@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
>      +  */
>      + void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
>      + 
>      ++char *fsmonitor_resolve_alias(const char *path,
>      ++	struct alias_info *alias);
>      ++
>      + #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
>      + #endif /* FSMONITOR_DAEMON_H */
>      +
>      + ## fsmonitor-path-utils.h ##
>      +@@
>      + #ifndef FSM_PATH_UTILS_H
>      + #define FSM_PATH_UTILS_H
>        
>      - 		case IS_WORKDIR_PATH:
>      - 			/* try to queue normal pathnames */
>      --
>      - 			if (trace_pass_fl(&trace_fsmonitor))
>      - 				log_flags_set(path_k, event_flags[k]);
>      ++#include "strbuf.h"
>      ++
>      ++struct alias_info
>      ++{
>      ++	struct strbuf alias;
>      ++	struct strbuf points_to;
>      ++};
>      ++
>      + struct fs_info {
>      + 	int is_remote;
>      + 	char *typename;
>      +@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
>      +  */
>      + int fsmonitor__is_fs_remote(const char *path);
>        
>      ++/*
>      ++ * Get the alias in given path, if any.
>      ++ *
>      ++ * Sets alias to the first alias that matches any part of the path.
>      ++ *
>      ++ * Returns -1 on error, 0 otherwise.
>      ++ */
>      ++int fsmonitor__get_alias(const char *path, struct alias_info *info);
>      ++
>      ++/*
>      ++ * Resolve the path against the given alias.
>      ++ *
>      ++ * Returns the resolved path if there is one, NULL otherwise.
>      ++ */
>      ++char *fsmonitor__resolve_alias(const char *path,
>      ++	const struct alias_info *info);
>      ++
>      ++
>      + #endif

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                           ` (4 preceding siblings ...)
  2022-09-12 15:27         ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
@ 2022-09-12 15:35         ` Junio C Hamano
  2022-09-12 19:35           ` Eric DeCosta
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
  6 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-12 15:35 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Eric DeCosta (4):
>   fsmonitor: refactor filesystem checks to common interface
>   fsmonitor: relocate socket file if .git directory is remote
>   fsmonitor: avoid socket location check if using hook
>   fsmonitor: deal with synthetic firmlinks on macOS
>
>  Makefile                                 |   2 +
>  builtin/fsmonitor--daemon.c              |   8 ++
>  compat/fsmonitor/fsm-ipc-darwin.c        |  46 ++++++++
>  compat/fsmonitor/fsm-ipc-win32.c         |   4 +
>  compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
>  compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++
>  compat/fsmonitor/fsm-path-utils-win32.c  | 106 ++++++++++++++++++
>  compat/fsmonitor/fsm-settings-darwin.c   |  70 ++++--------
>  compat/fsmonitor/fsm-settings-win32.c    | 106 +-----------------
>  contrib/buildsystems/CMakeLists.txt      |   4 +
>  fsmonitor--daemon.h                      |   6 ++
>  fsmonitor-ipc.c                          |   2 -
>  fsmonitor-path-utils.h                   |  49 +++++++++
>  fsmonitor-settings.c                     |  58 +++++++++-
>  fsmonitor-settings.h                     |   2 +-
>  15 files changed, 441 insertions(+), 159 deletions(-)
>  create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
>  create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
>  create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
>  create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
>  create mode 100644 fsmonitor-path-utils.h
>
>
> base-commit: be1a02a17ede4082a86dfbfee0f54f345e8b43ac

Another curious thing I need your help on.

The cover letter says that this work is a follow-up to extend the
previous work for Windows, and I would have expected these patches
to build on 85dc0da6 (fsmonitor: option to allow fsmonitor to run
against network-mounted repos, 2022-08-11), which is not yet in
be1a02a1 (The seventeenth batch, 2022-09-01).

What is going on?

Thanks.


^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-12 15:35         ` Junio C Hamano
@ 2022-09-12 19:35           ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-12 19:35 UTC (permalink / raw)
  To: Junio C Hamano, Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano
> Sent: Monday, September 12, 2022 11:36 AM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>
> Cc: git@vger.kernel.org; Jeff Hostetler <git@jeffhostetler.com>; Eric Sunshine
> <sunshine@sunshineco.com>; Torsten Bögershausen <tboegi@web.de>;
> Ævar Arnfjörð Bjarmason <avarab@gmail.com>; Ramsay Jones
> <ramsay@ramsayjones.plus.com>; Johannes Schindelin
> <Johannes.Schindelin@gmx.de>; Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run
> against network-mounted repos
> 
> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
> > Eric DeCosta (4):
> >   fsmonitor: refactor filesystem checks to common interface
> >   fsmonitor: relocate socket file if .git directory is remote
> >   fsmonitor: avoid socket location check if using hook
> >   fsmonitor: deal with synthetic firmlinks on macOS
> >
> >  Makefile                                 |   2 +
> >  builtin/fsmonitor--daemon.c              |   8 ++
> >  compat/fsmonitor/fsm-ipc-darwin.c        |  46 ++++++++
> >  compat/fsmonitor/fsm-ipc-win32.c         |   4 +
> >  compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
> >  compat/fsmonitor/fsm-path-utils-darwin.c | 131
> > +++++++++++++++++++++++  compat/fsmonitor/fsm-path-utils-win32.c  |
> 106 ++++++++++++++++++
> >  compat/fsmonitor/fsm-settings-darwin.c   |  70 ++++--------
> >  compat/fsmonitor/fsm-settings-win32.c    | 106 +-----------------
> >  contrib/buildsystems/CMakeLists.txt      |   4 +
> >  fsmonitor--daemon.h                      |   6 ++
> >  fsmonitor-ipc.c                          |   2 -
> >  fsmonitor-path-utils.h                   |  49 +++++++++
> >  fsmonitor-settings.c                     |  58 +++++++++-
> >  fsmonitor-settings.h                     |   2 +-
> >  15 files changed, 441 insertions(+), 159 deletions(-)  create mode
> > 100644 compat/fsmonitor/fsm-ipc-darwin.c  create mode 100644
> > compat/fsmonitor/fsm-ipc-win32.c  create mode 100644
> > compat/fsmonitor/fsm-path-utils-darwin.c
> >  create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
> >  create mode 100644 fsmonitor-path-utils.h
> >
> >
> > base-commit: be1a02a17ede4082a86dfbfee0f54f345e8b43ac
> 
> Another curious thing I need your help on.
> 
> The cover letter says that this work is a follow-up to extend the previous
> work for Windows, and I would have expected these patches to build on
> 85dc0da6 (fsmonitor: option to allow fsmonitor to run against network-
> mounted repos, 2022-08-11), which is not yet in
> be1a02a1 (The seventeenth batch, 2022-09-01).
> 
> What is going on?
> 
> Thanks.

I've been rolling that around in the back of my head over the last couple of days
actually.  Initially I though the macOS changes were going to be independent of
my earlier changes, but with all the refactoring that has occurred, that is no
longer the case. I've rebased my current patch set on top of those earlier changes.
v6 will incorporate that, the review feedback that I've received so far and add doc
for the new config options. I'll be sure to note all of this in the cover letter for v6.

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-12 15:27         ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
@ 2022-09-12 19:37           ` Junio C Hamano
  2022-09-12 19:39             ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-12 19:37 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Junio C Hamano <gitster@pobox.com> writes:

>> Eric DeCosta (4):
>>   fsmonitor: refactor filesystem checks to common interface
>>   fsmonitor: relocate socket file if .git directory is remote
>>   fsmonitor: avoid socket location check if using hook
>>   fsmonitor: deal with synthetic firmlinks on macOS
>
> It looks like this one is organized quite differently from the last
> iteration <pull.1326.v4.git.1661962145.gitgitgadget@gmail.com>

Given that we are very close to -rc period, I am very tempted to say
that this follow-up effort can and should be separated out of the
previous "windows only" one, and we should merge the single patch
that has been in 'next' to 'master' without waiting for this "macOS,
too" topic.  What do people think?

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-12 19:37           ` Junio C Hamano
@ 2022-09-12 19:39             ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-12 19:39 UTC (permalink / raw)
  To: Junio C Hamano, Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano
> Sent: Monday, September 12, 2022 3:38 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>
> Cc: git@vger.kernel.org; Jeff Hostetler <git@jeffhostetler.com>; Eric Sunshine
> <sunshine@sunshineco.com>; Torsten Bögershausen <tboegi@web.de>;
> Ævar Arnfjörð Bjarmason <avarab@gmail.com>; Ramsay Jones
> <ramsay@ramsayjones.plus.com>; Johannes Schindelin
> <Johannes.Schindelin@gmx.de>; Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run
> against network-mounted repos
> 
> Junio C Hamano <gitster@pobox.com> writes:
> 
> >> Eric DeCosta (4):
> >>   fsmonitor: refactor filesystem checks to common interface
> >>   fsmonitor: relocate socket file if .git directory is remote
> >>   fsmonitor: avoid socket location check if using hook
> >>   fsmonitor: deal with synthetic firmlinks on macOS
> >
> > It looks like this one is organized quite differently from the last
> > iteration <pull.1326.v4.git.1661962145.gitgitgadget@gmail.com>
> 
> Given that we are very close to -rc period, I am very tempted to say that this
> follow-up effort can and should be separated out of the previous "windows
> only" one, and we should merge the single patch that has been in 'next' to
> 'master' without waiting for this "macOS, too" topic.  What do people think?
> 
> Thanks.

I'm totally fine with that, turned out to be a bit more involved than I initially anticipated.

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-10 20:00       ` [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                           ` (5 preceding siblings ...)
  2022-09-12 15:35         ` Junio C Hamano
@ 2022-09-13 20:27         ` Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 1/6] " Eric DeCosta via GitGitGadget
                             ` (7 more replies)
  6 siblings, 8 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

v6 differs from v5:

 * incorporates earlier, Windows-specific changes that have not made it back
   yet to the master branch
 * incorporates code review feedback
 * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'

v5 differs significantly from earlier versions:

 * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
   such that these options are no longer added to the settings data
   structure but are rather read from config at point of use
 * refactoring of code for handling platform-specific file system checks via
   a common interface to avoid platform #ifdef in IPC code and be in-model
   with other platform-specific fsmonitor code
 * dealing with 'synthetic firmlinks' on macOS

Eric DeCosta (6):
  fsmonitor: option to allow fsmonitor to run against network-mounted
    repos
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: add documentation for allowRemote and socketDir options

 Documentation/git-fsmonitor--daemon.txt  |  35 ++++++
 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |   8 ++
 compat/fsmonitor/fsm-ipc-darwin.c        |  46 +++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   4 +
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 145 +++++++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  70 ++++-------
 compat/fsmonitor/fsm-settings-win32.c    | 106 +----------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   6 +
 fsmonitor-ipc.c                          |   2 -
 fsmonitor-path-utils.h                   |  59 +++++++++
 fsmonitor-settings.c                     |  58 ++++++++-
 fsmonitor-settings.h                     |   2 +-
 16 files changed, 526 insertions(+), 159 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: dd3f6c4cae7e3b15ce984dce8593ff7569650e24
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v5:

 -:  ----------- > 1:  3233c908c4a fsmonitor: option to allow fsmonitor to run against network-mounted repos
 1:  2f647b53e4d ! 2:  d2a8fc6b707 fsmonitor: refactor filesystem checks to common interface
     @@ compat/fsmonitor/fsm-path-utils-win32.c (new)
      +#include "fsmonitor-path-utils.h"
      +
      +/*
     ++ * Check remote working directory protocol.
     ++ *
     ++ * Return -1 if client machine cannot get remote protocol information.
     ++ */
     ++static int check_remote_protocol(wchar_t *wpath)
     ++{
     ++	HANDLE h;
     ++	FILE_REMOTE_PROTOCOL_INFO proto_info;
     ++
     ++	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
     ++			FILE_FLAG_BACKUP_SEMANTICS, NULL);
     ++
     ++	if (h == INVALID_HANDLE_VALUE) {
     ++		error(_("[GLE %ld] unable to open for read '%ls'"),
     ++		      GetLastError(), wpath);
     ++		return -1;
     ++	}
     ++
     ++	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
     ++		&proto_info, sizeof(proto_info))) {
     ++		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
     ++		      GetLastError(), wpath);
     ++		CloseHandle(h);
     ++		return -1;
     ++	}
     ++
     ++	CloseHandle(h);
     ++
     ++	trace_printf_key(&trace_fsmonitor,
     ++				"check_remote_protocol('%ls') remote protocol %#8.8lx",
     ++				wpath, proto_info.Protocol);
     ++
     ++	return 0;
     ++}
     ++
     ++/*
      + * Notes for testing:
      + *
      + * (a) Windows allows a network share to be mapped to a drive letter.
     @@ compat/fsmonitor/fsm-path-utils-win32.c (new)
      +			 "DriveType '%s' L'%ls' (%u)",
      +			 path, wfullpath, driveType);
      +
     -+	if (driveType == DRIVE_REMOTE)
     ++	if (driveType == DRIVE_REMOTE) {
      +		fs_info->is_remote = 1;
     -+	else
     ++		if (check_remote_protocol(wfullpath) < 0)
     ++			return -1;
     ++	} else {
      +		fs_info->is_remote = 0;
     ++	}
      +
      +	trace_printf_key(&trace_fsmonitor,
      +				"'%s' is_remote: %d",
     @@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
       	return FSMONITOR_REASON_OK;
       }
       
     +-/*
     +- * Check if monitoring remote working directories is allowed.
     +- *
     +- * By default, monitoring remote working directories is
     +- * disabled.  Users may override this behavior in enviroments where
     +- * they have proper support.
     +- */
     +-static int check_config_allowremote(struct repository *r)
     +-{
     +-	int allow;
     +-
     +-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
     +-		return allow;
     +-
     +-	return -1; /* fsmonitor.allowremote not set */
     +-}
     +-
     +-/*
     +- * Check remote working directory protocol.
     +- *
     +- * Error if client machine cannot get remote protocol information.
     +- */
     +-static int check_remote_protocol(wchar_t *wpath)
     +-{
     +-	HANDLE h;
     +-	FILE_REMOTE_PROTOCOL_INFO proto_info;
     +-
     +-	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
     +-			FILE_FLAG_BACKUP_SEMANTICS, NULL);
     +-
     +-	if (h == INVALID_HANDLE_VALUE) {
     +-		error(_("[GLE %ld] unable to open for read '%ls'"),
     +-		      GetLastError(), wpath);
     +-		return -1;
     +-	}
     +-
     +-	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
     +-		&proto_info, sizeof(proto_info))) {
     +-		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
     +-		      GetLastError(), wpath);
     +-		CloseHandle(h);
     +-		return -1;
     +-	}
     +-
     +-	CloseHandle(h);
     +-
     +-	trace_printf_key(&trace_fsmonitor,
     +-				"check_remote_protocol('%ls') remote protocol %#8.8lx",
     +-				wpath, proto_info.Protocol);
     +-
     +-	return 0;
     +-}
     +-
      -/*
      - * Remote working directories are problematic for FSMonitor.
      - *
     @@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
      - */
      -static enum fsmonitor_reason check_remote(struct repository *r)
      -{
     +-	int ret;
      -	wchar_t wpath[MAX_PATH];
      -	wchar_t wfullpath[MAX_PATH];
      -	size_t wlen;
     @@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
      -		trace_printf_key(&trace_fsmonitor,
      -				 "check_remote('%s') true",
      -				 r->worktree);
     +-
     +-		ret = check_remote_protocol(wfullpath);
     +-		if (ret < 0)
     +-			return FSMONITOR_REASON_ERROR;
     +-
     +-		switch (check_config_allowremote(r)) {
     +-		case 0: /* config overrides and disables */
     +-			return FSMONITOR_REASON_REMOTE;
     +-		case 1: /* config overrides and enables */
     +-			return FSMONITOR_REASON_OK;
     +-		default:
     +-			break; /* config has no opinion */
     +-		}
     +-
      -		return FSMONITOR_REASON_REMOTE;
      -	}
      -
 2:  28d08bcf808 = 3:  edef029a298 fsmonitor: relocate socket file if .git directory is remote
 3:  ff54b6e0bb5 = 4:  3428bcf8763 fsmonitor: avoid socket location check if using hook
 4:  b7d6cf44695 ! 5:  9c1f408ae6d fsmonitor: deal with synthetic firmlinks on macOS
     @@ Commit message
          ability to create symlinks in the root directory and replaced them with
          'synthetic firmlinks'. See 'man synthetic.conf'
      
     -    When FSEevents reports the path of changed files, if the path invloves
     +    When FSEevents reports the path of changed files, if the path involves
          a synthetic firmlink, the path is reported from the point of the
          synthetic firmlink and not the real path. For example:
      
     @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
       	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
       	state.nr_paths_watching = 1;
       
     -+	strbuf_init(&state.alias.alias, 0);
     -+	strbuf_init(&state.alias.points_to, 0);
     ++	state.alias.alias = NULL;
     ++	state.alias.points_to = NULL;
      +	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
      +		err = error(_("could not get worktree alias"));
      +		goto done;
     @@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
      + * information, but for now there is only one caller of this function.
      + *
      + * If there is more than one alias for the path, that is another
     -+ * matter altogteher.
     ++ * matter altogether.
      + */
      +int fsmonitor__get_alias(const char *path, struct alias_info *info)
      +{
     @@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
      +			strbuf_setlen(&points_to, read);
      +			if ((strncmp(points_to.buf, path, points_to.len) == 0)
      +				&& path[points_to.len] == '/') {
     -+				strbuf_addbuf(&info->alias, &alias);
     -+				strbuf_addbuf(&info->points_to, &points_to);
     ++				info->alias = strbuf_detach(&alias, NULL);
     ++				info->points_to = strbuf_detach(&points_to, NULL);
      +				trace_printf_key(&trace_fsmonitor,
      +					"Found alias for '%s' : '%s' -> '%s'",
     -+					path, info->alias.buf, info->points_to.buf);
     ++					path, info->alias, info->points_to);
      +				retval = 0;
      +				goto done;
      +			}
     @@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
      +char *fsmonitor__resolve_alias(const char *path,
      +	const struct alias_info *info)
      +{
     -+	int len = info->alias.len;
     ++	int len = info->alias ? strlen(info->alias) : 0;
      +
      +	if (!len)
      +		return NULL;
      +
     -+	if ((strncmp(info->alias.buf, path, len) == 0)
     ++	if ((strncmp(info->alias, path, len) == 0)
      +		&& path[len] == '/') {
      +		struct strbuf tmp;
      +		const char *remainder = path + len;
     ++		int ptr_len = strlen(info->points_to);
      +		int rem_len = strlen(remainder);
      +
     -+		strbuf_init(&tmp, info->points_to.len + rem_len);
     -+		strbuf_addbuf(&tmp, &info->points_to);
     ++		strbuf_init(&tmp, ptr_len + rem_len);
     ++		strbuf_add(&tmp, info->points_to, ptr_len);
      +		strbuf_add(&tmp, remainder, rem_len);
      +		return strbuf_detach(&tmp, NULL);
      +	}
     @@ fsmonitor-path-utils.h
      +
      +struct alias_info
      +{
     -+	struct strbuf alias;
     -+	struct strbuf points_to;
     ++	char *alias;
     ++	char *points_to;
      +};
      +
       struct fs_info {
     @@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_i
      + *
      + * Sets alias to the first alias that matches any part of the path.
      + *
     ++ * If an alias is found, info.alias and info.points_to are set to the
     ++ * found mapping.
     ++ *
      + * Returns -1 on error, 0 otherwise.
     ++ *
     ++ * The caller owns the storage that is occupied by set info.alias and
     ++ * info.points_to and is responsible for releasing it with `free(3)`
     ++ * when done.
      + */
      +int fsmonitor__get_alias(const char *path, struct alias_info *info);
      +
     @@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_i
      + * Resolve the path against the given alias.
      + *
      + * Returns the resolved path if there is one, NULL otherwise.
     ++ *
     ++ * The caller owns the storage that the returned string occupies and
     ++ * is responsible for releasing it with `free(3)` when done.
      + */
      +char *fsmonitor__resolve_alias(const char *path,
      +	const struct alias_info *info);
 -:  ----------- > 6:  d2c95e34d3a fsmonitor: add documentation for allowRemote and socketDir options

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v6 1/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 2/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                             ` (6 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Though perhaps not common, there are use cases where users have large,
network-mounted repos. Having the ability to run fsmonitor against
network paths would benefit those users.

Most modern Samba-based filers have the necessary support to enable
fsmonitor on network-mounted repos. As a first step towards enabling
fsmonitor to work against network-mounted repos, introduce a
configuration option, 'fsmonitor.allowRemote'. Setting this option to
true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-win32.c | 68 +++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index 907655720bb..e5ec5b0a9f7 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
+/*
+ * Check if monitoring remote working directories is allowed.
+ *
+ * By default, monitoring remote working directories is
+ * disabled.  Users may override this behavior in enviroments where
+ * they have proper support.
+ */
+static int check_config_allowremote(struct repository *r)
+{
+	int allow;
+
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		return allow;
+
+	return -1; /* fsmonitor.allowremote not set */
+}
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Error if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
 /*
  * Remote working directories are problematic for FSMonitor.
  *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
  */
 static enum fsmonitor_reason check_remote(struct repository *r)
 {
+	int ret;
 	wchar_t wpath[MAX_PATH];
 	wchar_t wfullpath[MAX_PATH];
 	size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 		trace_printf_key(&trace_fsmonitor,
 				 "check_remote('%s') true",
 				 r->worktree);
+
+		ret = check_remote_protocol(wfullpath);
+		if (ret < 0)
+			return FSMONITOR_REASON_ERROR;
+
+		switch (check_config_allowremote(r)) {
+		case 0: /* config overrides and disables */
+			return FSMONITOR_REASON_REMOTE;
+		case 1: /* config overrides and enables */
+			return FSMONITOR_REASON_OK;
+		default:
+			break; /* config has no opinion */
+		}
+
 		return FSMONITOR_REASON_REMOTE;
 	}
 
-- 
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
+/*
+ * Check if monitoring remote working directories is allowed.
+ *
+ * By default, monitoring remote working directories is
+ * disabled.  Users may override this behavior in enviroments where
+ * they have proper support.
+ */
+static int check_config_allowremote(struct repository *r)
+{
+	int allow;
+
+	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+		return allow;
+
+	return -1; /* fsmonitor.allowremote not set */
+}
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Error if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
 /*
  * Remote working directories are problematic for FSMonitor.
  *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
  */
 static enum fsmonitor_reason check_remote(struct repository *r)
 {
+	int ret;
 	wchar_t wpath[MAX_PATH];
 	wchar_t wfullpath[MAX_PATH];
 	size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 		trace_printf_key(&trace_fsmonitor,
 				 "check_remote('%s') true",
 				 r->worktree);
+
+		ret = check_remote_protocol(wfullpath);
+		if (ret < 0)
+			return FSMONITOR_REASON_ERROR;
+
+		switch (check_config_allowremote(r)) {
+		case 0: /* config overrides and disables */
+			return FSMONITOR_REASON_REMOTE;
+		case 1: /* config overrides and enables */
+			return FSMONITOR_REASON_OK;
+		default:
+			break; /* config has no opinion */
+		}
+
 		return FSMONITOR_REASON_REMOTE;
 	}
 
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v6 2/6] fsmonitor: refactor filesystem checks to common interface
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 1/6] " Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
                             ` (5 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Provide a common interface for getting basic filesystem information
including filesystem type and whether the filesystem is remote.

Refactor existing code for getting basic filesystem info and detecting
remote file systems to the new interface.

Refactor filesystem checks to leverage new interface. For macOS,
error-out if the Unix Domain socket (UDS) file is on a remote
filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                                 |   1 +
 compat/fsmonitor/fsm-path-utils-darwin.c |  40 ++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 128 +++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  62 +++-----
 compat/fsmonitor/fsm-settings-win32.c    | 172 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   2 +
 fsmonitor-path-utils.h                   |  23 +++
 fsmonitor-settings.c                     |  50 +++++++
 8 files changed, 263 insertions(+), 215 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h

diff --git a/Makefile b/Makefile
index c6e126e54c2..b026f3e8cf0 100644
--- a/Makefile
+++ b/Makefile
@@ -2038,6 +2038,7 @@ endif
 ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
 ifeq ($(TCLTK_PATH),)
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
new file mode 100644
index 00000000000..067cbe6990a
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -0,0 +1,40 @@
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+#include <sys/param.h>
+#include <sys/mount.h>
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	struct statfs fs;
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return -1;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
+	if (!(fs.f_flags & MNT_LOCAL))
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	fs_info->typename = fs.f_fstypename;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
new file mode 100644
index 00000000000..a90b8f7925b
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -0,0 +1,128 @@
+#include "cache.h"
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Return -1 if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
+/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
+ *     (This is the normal method to access it.)
+ *
+ *     $ NET USE Z: \\server\share
+ *     $ git -C Z:/repo status
+ *
+ * (b) Windows allows a network share to be referenced WITHOUT mapping
+ *     it to drive letter.
+ *
+ *     $ NET USE \\server\share\dir
+ *     $ git -C //server/share/repo status
+ *
+ * (c) Windows allows "SUBST" to create a fake drive mapping to an
+ *     arbitrary path (which may be remote)
+ *
+ *     $ SUBST Q: Z:\repo
+ *     $ git -C Q:/ status
+ *
+ * (d) Windows allows a directory symlink to be created on a local
+ *     file system that points to a remote repo.
+ *
+ *     $ mklink /d ./link //server/share/repo
+ *     $ git -C ./link status
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	wchar_t wpath[MAX_PATH];
+	wchar_t wfullpath[MAX_PATH];
+	size_t wlen;
+	UINT driveType;
+
+	/*
+	 * Do everything in wide chars because the drive letter might be
+	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
+	 */
+	if (xutftowcs_path(wpath, path) < 0) {
+		return -1;
+	}
+
+	/*
+	 * GetDriveTypeW() requires a final slash.  We assume that the
+	 * worktree pathname points to an actual directory.
+	 */
+	wlen = wcslen(wpath);
+	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
+		wpath[wlen++] = L'\\';
+		wpath[wlen] = 0;
+	}
+
+	/*
+	 * Normalize the path.  If nothing else, this converts forward
+	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
+	 * correctly handle some UNC "\\server\share\..." paths.
+	 */
+	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) {
+		return -1;
+	}
+
+	driveType = GetDriveTypeW(wfullpath);
+	trace_printf_key(&trace_fsmonitor,
+			 "DriveType '%s' L'%ls' (%u)",
+			 path, wfullpath, driveType);
+
+	if (driveType == DRIVE_REMOTE) {
+		fs_info->is_remote = 1;
+		if (check_remote_protocol(wfullpath) < 0)
+			return -1;
+	} else {
+		fs_info->is_remote = 0;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dba3ced6bb7 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -1,32 +1,10 @@
-#include "cache.h"
 #include "config.h"
-#include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
-#include <sys/param.h>
-#include <sys/mount.h>
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
-/*
- * [1] Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type (NFS, SAMBA, etc.) dictates whether notification events
- * are available at all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
+ /*
  * For the builtin FSMonitor, we create the Unix domain socket for the
  * IPC in the .git directory.  If the working directory is remote,
  * then the socket will be created on the remote file system.  This
@@ -38,40 +16,34 @@
  * be taken to ensure that $HOME is actually local and not a managed
  * file share.)
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
+ * FAT32 and NTFS working directories are problematic too.
  *
  * The builtin FSMonitor uses a Unix domain socket in the .git
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
  */
-static enum fsmonitor_reason check_volume(struct repository *r)
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
-	struct statfs fs;
+	struct fs_info fs;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	struct strbuf path = STRBUF_INIT;
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
-	if (statfs(r->worktree, &fs) == -1) {
-		int saved_errno = errno;
-		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-				 r->worktree, strerror(saved_errno));
-		errno = saved_errno;
+	if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) {
+		strbuf_release(&path);
 		return FSMONITOR_REASON_ERROR;
 	}
 
-	trace_printf_key(&trace_fsmonitor,
-			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
 
-	if (!(fs.f_flags & MNT_LOCAL))
+	if (fs.is_remote)
 		return FSMONITOR_REASON_REMOTE;
 
-	if (!strcmp(fs.f_fstypename, "msdos")) /* aka FAT32 */
+	if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
 		return FSMONITOR_REASON_NOSOCKETS;
 
-	if (!strcmp(fs.f_fstypename, "ntfs"))
+	if (!strcmp(fs.typename, "ntfs"))
 		return FSMONITOR_REASON_NOSOCKETS;
 
 	return FSMONITOR_REASON_OK;
@@ -81,7 +53,7 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_volume(r);
+	reason = check_uds_volume(r);
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index e5ec5b0a9f7..d88b06ae610 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -1,8 +1,9 @@
 #include "cache.h"
 #include "config.h"
 #include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * VFS for Git is incompatible with FSMonitor.
@@ -24,171 +25,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
-/*
- * Check remote working directory protocol.
- *
- * Error if client machine cannot get remote protocol information.
- */
-static int check_remote_protocol(wchar_t *wpath)
-{
-	HANDLE h;
-	FILE_REMOTE_PROTOCOL_INFO proto_info;
-
-	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
-			FILE_FLAG_BACKUP_SEMANTICS, NULL);
-
-	if (h == INVALID_HANDLE_VALUE) {
-		error(_("[GLE %ld] unable to open for read '%ls'"),
-		      GetLastError(), wpath);
-		return -1;
-	}
-
-	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
-		&proto_info, sizeof(proto_info))) {
-		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
-		      GetLastError(), wpath);
-		CloseHandle(h);
-		return -1;
-	}
-
-	CloseHandle(h);
-
-	trace_printf_key(&trace_fsmonitor,
-				"check_remote_protocol('%ls') remote protocol %#8.8lx",
-				wpath, proto_info.Protocol);
-
-	return 0;
-}
-
-/*
- * Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type dictates whether notification events are available at
- * all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- * Notes for testing:
- *
- * (a) Windows allows a network share to be mapped to a drive letter.
- *     (This is the normal method to access it.)
- *
- *     $ NET USE Z: \\server\share
- *     $ git -C Z:/repo status
- *
- * (b) Windows allows a network share to be referenced WITHOUT mapping
- *     it to drive letter.
- *
- *     $ NET USE \\server\share\dir
- *     $ git -C //server/share/repo status
- *
- * (c) Windows allows "SUBST" to create a fake drive mapping to an
- *     arbitrary path (which may be remote)
- *
- *     $ SUBST Q: Z:\repo
- *     $ git -C Q:/ status
- *
- * (d) Windows allows a directory symlink to be created on a local
- *     file system that points to a remote repo.
- *
- *     $ mklink /d ./link //server/share/repo
- *     $ git -C ./link status
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
-	int ret;
-	wchar_t wpath[MAX_PATH];
-	wchar_t wfullpath[MAX_PATH];
-	size_t wlen;
-	UINT driveType;
-
-	/*
-	 * Do everything in wide chars because the drive letter might be
-	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
-	 */
-	if (xutftowcs_path(wpath, r->worktree) < 0)
-		return FSMONITOR_REASON_ERROR;
-
-	/*
-	 * GetDriveTypeW() requires a final slash.  We assume that the
-	 * worktree pathname points to an actual directory.
-	 */
-	wlen = wcslen(wpath);
-	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
-		wpath[wlen++] = L'\\';
-		wpath[wlen] = 0;
-	}
-
-	/*
-	 * Normalize the path.  If nothing else, this converts forward
-	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
-	 * correctly handle some UNC "\\server\share\..." paths.
-	 */
-	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
-		return FSMONITOR_REASON_ERROR;
-
-	driveType = GetDriveTypeW(wfullpath);
-	trace_printf_key(&trace_fsmonitor,
-			 "DriveType '%s' L'%ls' (%u)",
-			 r->worktree, wfullpath, driveType);
-
-	if (driveType == DRIVE_REMOTE) {
-		trace_printf_key(&trace_fsmonitor,
-				 "check_remote('%s') true",
-				 r->worktree);
-
-		ret = check_remote_protocol(wfullpath);
-		if (ret < 0)
-			return FSMONITOR_REASON_ERROR;
-
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
-	}
-
-	return FSMONITOR_REASON_OK;
-}
-
 enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
@@ -197,9 +33,5 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
-	reason = check_remote(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
-
 	return FSMONITOR_REASON_OK;
 }
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..b88494bf59b 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
new file mode 100644
index 00000000000..e48592887e7
--- /dev/null
+++ b/fsmonitor-path-utils.h
@@ -0,0 +1,23 @@
+#ifndef FSM_PATH_UTILS_H
+#define FSM_PATH_UTILS_H
+
+struct fs_info {
+	int is_remote;
+	char *typename;
+};
+
+/*
+ * Get some basic filesystem informtion for the given path
+ *
+ * Returns -1 on error, zero otherwise.
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+
+/*
+ * Determines if the filesystem that path resides on is remote.
+ *
+ * Returns -1 on error, 0 if not remote, 1 if remote.
+ */
+int fsmonitor__is_fs_remote(const char *path);
+
+#endif
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..d288cbad479 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 1/6] " Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 2/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-14  0:48             ` Junio C Hamano
  2022-09-13 20:27           ` [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
                             ` (4 subsequent siblings)
  7 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If the .git directory is on a remote file system, create the socket
file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                            |  1 +
 compat/fsmonitor/fsm-ipc-darwin.c   | 46 +++++++++++++++++++++++++++++
 compat/fsmonitor/fsm-ipc-win32.c    |  4 +++
 contrib/buildsystems/CMakeLists.txt |  2 ++
 fsmonitor-ipc.c                     |  2 --
 5 files changed, 53 insertions(+), 2 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c

diff --git a/Makefile b/Makefile
index b026f3e8cf0..5cd5ad818b8 100644
--- a/Makefile
+++ b/Makefile
@@ -2033,6 +2033,7 @@ ifdef FSMONITOR_DAEMON_BACKEND
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
 endif
 
 ifdef FSMONITOR_OS_SETTINGS
diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
new file mode 100644
index 00000000000..afaca96dab9
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-darwin.c
@@ -0,0 +1,46 @@
+#include "cache.h"
+#include "config.h"
+#include "strbuf.h"
+#include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
+	static const char *ipc_path;
+	SHA_CTX sha1ctx;
+	char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (ipc_path)
+		return ipc_path;
+
+	ipc_path = fsmonitor_ipc__get_default_path();
+
+	/* By default the socket file is created in the .git directory */
+	if (fsmonitor__is_fs_remote(ipc_path) < 1)
+		return ipc_path;
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
+	SHA1_Final(hash, &sha1ctx);
+
+	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
+
+	/* Create the socket file in either socketDir or $HOME */
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+
+	ipc_path = interpolate_path(ipc_file.buf, 1);
+	if (!ipc_path)
+		die(_("Invalid path: %s"), ipc_file.buf);
+
+	strbuf_release(&ipc_file);
+	return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
new file mode 100644
index 00000000000..769a88639f6
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-win32.c
@@ -0,0 +1,4 @@
+#include "cache.h"
+#include "fsmonitor-ipc.h"
+
+GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index b88494bf59b..7e7b6b9a362 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..caad2e246a0 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
 	return ipc_get_active_state(fsmonitor_ipc__get_path());
-- 
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
 	return ipc_get_active_state(fsmonitor_ipc__get_path());
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
                             ` (2 preceding siblings ...)
  2022-09-13 20:27           ` [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-14  1:48             ` Junio C Hamano
  2022-09-13 20:27           ` [PATCH v6 5/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
                             ` (3 subsequent siblings)
  7 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If monitoring is done via fsmonitor hook rather than IPC there is no
need to check if the location of the Unix Domain socket (UDS) file is
on a remote filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
 compat/fsmonitor/fsm-settings-win32.c  |  2 +-
 fsmonitor-settings.c                   |  8 ++++----
 fsmonitor-settings.h                   |  2 +-
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index dba3ced6bb7..3463c71763e 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_uds_volume(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
+	if (ipc) {
+		reason = check_uds_volume(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
+	}
 
 	return FSMONITOR_REASON_OK;
 }
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index d88b06ae610..a8af31b71de 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index d288cbad479..531a1b6f956 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 }
 #endif
 
-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
 {
 	if (!r->worktree) {
 		/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 		reason = check_remote(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-		reason = fsm_os__incompatible(r);
+		reason = fsm_os__incompatible(r, ipc);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
 	}
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..0721617b95a 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v6 5/6] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
                             ` (3 preceding siblings ...)
  2022-09-13 20:27           ` [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-13 20:27           ` [PATCH v6 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
                             ` (2 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     |  6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  6 ++
 fsmonitor-path-utils.h                   | 36 ++++++++++
 6 files changed, 164 insertions(+), 1 deletion(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 2c109cf8b37..32475962c92 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	state.alias.alias = NULL;
+	state.alias.points_to = NULL;
+	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+		err = error(_("could not get worktree alias"));
+		goto done;
+	}
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..179886bc15b 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (!path_k)
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index 067cbe6990a..13807f58e95 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -38,3 +41,92 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR * dir;
+	int read;
+	int retval;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to;
+
+	retval = 0;
+	dir = opendir("/");
+	if (!dir)
+		return -1;
+
+	strbuf_init(&alias, 256);
+	strbuf_init(&points_to, MAXPATHLEN);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addch(&alias, '/');
+		strbuf_add(&alias, de->d_name, strlen(de->d_name));
+
+		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
+		if (read > 0) {
+			strbuf_setlen(&points_to, read);
+			if ((strncmp(points_to.buf, path, points_to.len) == 0)
+				&& path[points_to.len] == '/') {
+				info->alias = strbuf_detach(&alias, NULL);
+				info->points_to = strbuf_detach(&points_to, NULL);
+				trace_printf_key(&trace_fsmonitor,
+					"Found alias for '%s' : '%s' -> '%s'",
+					path, info->alias, info->points_to);
+				retval = 0;
+				goto done;
+			}
+		} else if (errno != EINVAL) { /* Something other than not a link */
+			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
+			retval = -1;
+			goto done;
+		}
+	}
+
+	done:
+	closedir(dir);
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	int len = info->alias ? strlen(info->alias) : 0;
+
+	if (!len)
+		return NULL;
+
+	if ((strncmp(info->alias, path, len) == 0)
+		&& path[len] == '/') {
+		struct strbuf tmp;
+		const char *remainder = path + len;
+		int ptr_len = strlen(info->points_to);
+		int rem_len = strlen(remainder);
+
+		strbuf_init(&tmp, ptr_len + rem_len);
+		strbuf_add(&tmp, info->points_to, ptr_len);
+		strbuf_add(&tmp, remainder, rem_len);
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index a90b8f7925b..0d95bbb416f 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..98cbb430083 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
@@ -167,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,
  */
 void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
 
+char *fsmonitor_resolve_alias(const char *path,
+	struct alias_info *alias);
+
 #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
 #endif /* FSMONITOR_DAEMON_H */
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index e48592887e7..50ef37e57bb 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v6 6/6] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
                             ` (4 preceding siblings ...)
  2022-09-13 20:27           ` [PATCH v6 5/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-13 20:27           ` Eric DeCosta via GitGitGadget
  2022-09-16 17:58           ` [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Jeff Hostetler
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-13 20:27 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
Call-out experimental nature of 'fsmonitor.allowRemote' and limited file
system support for 'fsmonitor.socketDir'.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt
index cc142fb8612..0adccd0eced 100644
--- a/Documentation/git-fsmonitor--daemon.txt
+++ b/Documentation/git-fsmonitor--daemon.txt
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-13 20:27           ` [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-14  0:48             ` Junio C Hamano
  2022-09-14 15:47               ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-14  0:48 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +const char *fsmonitor_ipc__get_path(void)
> +{

Looks like a bit klunky API.  I would have expected it to take at
least the path to the worktree or a pointer to struct repository.

> +	static const char *ipc_path;
> +	SHA_CTX sha1ctx;
> +	char *sock_dir;
> +	struct strbuf ipc_file = STRBUF_INIT;
> +	unsigned char hash[SHA_DIGEST_LENGTH];
> +
> +	if (ipc_path)
> +		return ipc_path;
> +
> +	ipc_path = fsmonitor_ipc__get_default_path();
> +
> +	/* By default the socket file is created in the .git directory */
> +	if (fsmonitor__is_fs_remote(ipc_path) < 1)
> +		return ipc_path;
> +
> +	SHA1_Init(&sha1ctx);
> +	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
> +	SHA1_Final(hash, &sha1ctx);

I would not worry about SHA-1 hash collision for this use case, but
would worry more about .worktree possible being unique.

Can the .worktree string be aliased for the same directory in some
way (e.g. depending on the initialization sequence, can it be a full
pathname, a relative pathname, or can a pathname that looks like a
full-pathname have symbolic link component in it?) that ends up
creating two or more socket for the same worktree?

> +	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
> +
> +	/* Create the socket file in either socketDir or $HOME */
> +	if (sock_dir && *sock_dir)
> +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> +					sock_dir, hash_to_hex(hash));
> +	else
> +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
> +
> +	ipc_path = interpolate_path(ipc_file.buf, 1);
> +	if (!ipc_path)
> +		die(_("Invalid path: %s"), ipc_file.buf);
> +
> +	strbuf_release(&ipc_file);
> +	return ipc_path;
> +}

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook
  2022-09-13 20:27           ` [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-14  1:48             ` Junio C Hamano
  0 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-09-14  1:48 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Eric DeCosta <edecosta@mathworks.com>
>
> If monitoring is done via fsmonitor hook rather than IPC there is no
> need to check if the location of the Unix Domain socket (UDS) file is
> on a remote filesystem.

Nicely done.


>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
>  compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
>  compat/fsmonitor/fsm-settings-win32.c  |  2 +-
>  fsmonitor-settings.c                   |  8 ++++----
>  fsmonitor-settings.h                   |  2 +-
>  4 files changed, 12 insertions(+), 10 deletions(-)
>
> diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
> index dba3ced6bb7..3463c71763e 100644
> --- a/compat/fsmonitor/fsm-settings-darwin.c
> +++ b/compat/fsmonitor/fsm-settings-darwin.c
> @@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
>  	return FSMONITOR_REASON_OK;
>  }
>  
> -enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
> +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
>  {
>  	enum fsmonitor_reason reason;
>  
> -	reason = check_uds_volume(r);
> -	if (reason != FSMONITOR_REASON_OK)
> -		return reason;
> +	if (ipc) {
> +		reason = check_uds_volume(r);
> +		if (reason != FSMONITOR_REASON_OK)
> +			return reason;
> +	}
>  
>  	return FSMONITOR_REASON_OK;
>  }
> diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
> index d88b06ae610..a8af31b71de 100644
> --- a/compat/fsmonitor/fsm-settings-win32.c
> +++ b/compat/fsmonitor/fsm-settings-win32.c
> @@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
>  	return FSMONITOR_REASON_OK;
>  }
>  
> -enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
> +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
>  {
>  	enum fsmonitor_reason reason;
>  
> diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
> index d288cbad479..531a1b6f956 100644
> --- a/fsmonitor-settings.c
> +++ b/fsmonitor-settings.c
> @@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
>  }
>  #endif
>  
> -static enum fsmonitor_reason check_for_incompatible(struct repository *r)
> +static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
>  {
>  	if (!r->worktree) {
>  		/*
> @@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
>  		reason = check_remote(r);
>  		if (reason != FSMONITOR_REASON_OK)
>  			return reason;
> -		reason = fsm_os__incompatible(r);
> +		reason = fsm_os__incompatible(r, ipc);
>  		if (reason != FSMONITOR_REASON_OK)
>  			return reason;
>  	}
> @@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
>  
>  void fsm_settings__set_ipc(struct repository *r)
>  {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
>  
>  	if (reason != FSMONITOR_REASON_OK) {
>  		fsm_settings__set_incompatible(r, reason);
> @@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
>  
>  void fsm_settings__set_hook(struct repository *r, const char *path)
>  {
> -	enum fsmonitor_reason reason = check_for_incompatible(r);
> +	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
>  
>  	if (reason != FSMONITOR_REASON_OK) {
>  		fsm_settings__set_incompatible(r, reason);
> diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
> index d9c2605197f..0721617b95a 100644
> --- a/fsmonitor-settings.h
> +++ b/fsmonitor-settings.h
> @@ -48,7 +48,7 @@ struct fsmonitor_settings;
>   * fsm_os__* routines should considered private to fsm_settings__
>   * routines.
>   */
> -enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
> +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
>  #endif /* HAVE_FSMONITOR_OS_SETTINGS */
>  
>  #endif /* FSMONITOR_SETTINGS_H */

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-14  0:48             ` Junio C Hamano
@ 2022-09-14 15:47               ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-14 15:47 UTC (permalink / raw)
  To: Junio C Hamano, Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Junio C Hamano <gitster@pobox.com>
> Sent: Tuesday, September 13, 2022 8:48 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>
> Cc: git@vger.kernel.org; Jeff Hostetler <git@jeffhostetler.com>; Eric Sunshine
> <sunshine@sunshineco.com>; Torsten Bögershausen <tboegi@web.de>;
> Ævar Arnfjörð Bjarmason <avarab@gmail.com>; Ramsay Jones
> <ramsay@ramsayjones.plus.com>; Johannes Schindelin
> <Johannes.Schindelin@gmx.de>; Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is
> remote
> 
> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
> > +const char *fsmonitor_ipc__get_path(void) {
> 
> Looks like a bit klunky API.  I would have expected it to take at least the path
> to the worktree or a pointer to struct repository.
> 
OK, I'll change it to take a pointer to struct repository.

> > +	static const char *ipc_path;
> > +	SHA_CTX sha1ctx;
> > +	char *sock_dir;
> > +	struct strbuf ipc_file = STRBUF_INIT;
> > +	unsigned char hash[SHA_DIGEST_LENGTH];
> > +
> > +	if (ipc_path)
> > +		return ipc_path;
> > +
> > +	ipc_path = fsmonitor_ipc__get_default_path();
> > +
> > +	/* By default the socket file is created in the .git directory */
> > +	if (fsmonitor__is_fs_remote(ipc_path) < 1)
> > +		return ipc_path;
> > +
> > +	SHA1_Init(&sha1ctx);
> > +	SHA1_Update(&sha1ctx, the_repository->worktree,
> strlen(the_repository->worktree));
> > +	SHA1_Final(hash, &sha1ctx);
> 
> I would not worry about SHA-1 hash collision for this use case, but would
> worry more about .worktree possible being unique.
> 
> Can the .worktree string be aliased for the same directory in some way (e.g.
> depending on the initialization sequence, can it be a full pathname, a relative
> pathname, or can a pathname that looks like a full-pathname have symbolic
> link component in it?) that ends up creating two or more socket for the same
> worktree?
> 

.worktree is set to the real path and since hardlinks are not allowed across file systems, I think we are OK.

> > +	repo_config_get_string(the_repository, "fsmonitor.socketdir",
> > +&sock_dir);
> > +
> > +	/* Create the socket file in either socketDir or $HOME */
> > +	if (sock_dir && *sock_dir)
> > +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> > +					sock_dir, hash_to_hex(hash));
> > +	else
> > +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s",
> hash_to_hex(hash));
> > +
> > +	ipc_path = interpolate_path(ipc_file.buf, 1);
> > +	if (!ipc_path)
> > +		die(_("Invalid path: %s"), ipc_file.buf);
> > +
> > +	strbuf_release(&ipc_file);
> > +	return ipc_path;
> > +}


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
                             ` (5 preceding siblings ...)
  2022-09-13 20:27           ` [PATCH v6 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
@ 2022-09-16 17:58           ` Jeff Hostetler
  2022-09-16 20:02             ` Eric DeCosta
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
  7 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-16 17:58 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget, git
  Cc: Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta



On 9/13/22 4:27 PM, Eric DeCosta via GitGitGadget wrote:
> Follow-on to the work done to allow Windows to work against network-mounted
> repos for macOS.
> 
> Have macOS take advantage of the same configuration option,
> 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
> to true will override the default behavior (erroring-out) when a
> network-mounted repo is detected by fsmonitor.
> 
> The added wrinkle being that the Unix domain socket (UDS) file used for IPC
> cannot be created in a network location; instead $HOME is used if the
> default location is on the network. The user may, optionally, set the
> 'fsmonitor.socketDir' configuration option to a valid, local directory if
> $HOME itself is on the network or is simply not the desired location for the
> UDS file.
> 
> An additional issue is that for mount points in the root directory, FSEvents
> does not report a path that matches the worktree directory due to the
> introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
> to the worktree directory by interrogating the root filesystem for synthetic
> firmlinks and using that information to translate the path.
> 
> v6 differs from v5:
> 
>   * incorporates earlier, Windows-specific changes that have not made it back
>     yet to the master branch
>   * incorporates code review feedback
>   * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
> 
> v5 differs significantly from earlier versions:
> 
>   * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
>     such that these options are no longer added to the settings data
>     structure but are rather read from config at point of use
>   * refactoring of code for handling platform-specific file system checks via
>     a common interface to avoid platform #ifdef in IPC code and be in-model
>     with other platform-specific fsmonitor code
>   * dealing with 'synthetic firmlinks' on macOS
> 

I've looked at v5 and v6 and I like the direction this is heading,
so I'll mark this LGTM.  (I'm still traveling back from Git Merge,
so I haven't had time to test it out, but I think we should proceed
with it.)

Thanks for you patience and attention to detail on this!
Jeff


^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v7 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-13 20:27         ` [PATCH v6 0/6] " Eric DeCosta via GitGitGadget
                             ` (6 preceding siblings ...)
  2022-09-16 17:58           ` [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Jeff Hostetler
@ 2022-09-16 19:53           ` Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                               ` (7 more replies)
  7 siblings, 8 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

v7 differs from v6:

 * incorporates code review feedback

v6 differs from v5:

 * incorporates earlier, Windows-specific changes that have not made it back
   yet to the master branch
 * incorporates code review feedback
 * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'

v5 differs significantly from earlier versions:

 * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
   such that these options are no longer added to the settings data
   structure but are rather read from config at point of use
 * refactoring of code for handling platform-specific file system checks via
   a common interface to avoid platform #ifdef in IPC code and be in-model
   with other platform-specific fsmonitor code
 * dealing with 'synthetic firmlinks' on macOS

Eric DeCosta (6):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: add documentation for allowRemote and socketDir options

 Documentation/git-fsmonitor--daemon.txt  |  35 +++++
 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |  11 +-
 compat/fsmonitor/fsm-ipc-darwin.c        |  49 +++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   9 ++
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 145 +++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  70 +++------
 compat/fsmonitor/fsm-settings-win32.c    | 174 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   6 +
 fsmonitor-ipc.c                          |  18 ++-
 fsmonitor-ipc.h                          |   4 +-
 fsmonitor-path-utils.h                   |  59 ++++++++
 fsmonitor-settings.c                     |  58 +++++++-
 fsmonitor-settings.h                     |   2 +-
 17 files changed, 547 insertions(+), 237 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v6:

 1:  3233c908c4a < -:  ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos
 2:  d2a8fc6b707 = 1:  155a6890806 fsmonitor: refactor filesystem checks to common interface
 3:  edef029a298 ! 2:  075340bd2a7 fsmonitor: relocate socket file if .git directory is remote
     @@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND
       
       ifdef FSMONITOR_OS_SETTINGS
      
     + ## builtin/fsmonitor--daemon.c ##
     +@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
     + 	 * directory.)
     + 	 */
     + 	strbuf_init(&state.path_ipc, 0);
     +-	strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path()));
     ++	strbuf_addstr(&state.path_ipc,
     ++		absolute_path(fsmonitor_ipc__get_path(the_repository)));
     + 
     + 	/*
     + 	 * Confirm that we can create platform-specific resources for the
     +
       ## compat/fsmonitor/fsm-ipc-darwin.c (new) ##
      @@
      +#include "cache.h"
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +
      +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
      +
     -+const char *fsmonitor_ipc__get_path(void)
     ++const char *fsmonitor_ipc__get_path(struct repository *r)
      +{
      +	static const char *ipc_path;
      +	SHA_CTX sha1ctx;
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +	if (ipc_path)
      +		return ipc_path;
      +
     ++	if (!r)
     ++		r = the_repository;
     ++
      +	ipc_path = fsmonitor_ipc__get_default_path();
      +
      +	/* By default the socket file is created in the .git directory */
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +		return ipc_path;
      +
      +	SHA1_Init(&sha1ctx);
     -+	SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
     ++	SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
      +	SHA1_Final(hash, &sha1ctx);
      +
     -+	repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
     ++	repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
      +
      +	/* Create the socket file in either socketDir or $HOME */
      +	if (sock_dir && *sock_dir)
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      
       ## compat/fsmonitor/fsm-ipc-win32.c (new) ##
      @@
     -+#include "cache.h"
     ++#include "config.h"
      +#include "fsmonitor-ipc.h"
      +
     -+GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
     ++const char *fsmonitor_ipc__get_path(struct repository *r) {
     ++	static char *ret;
     ++	if (!ret)
     ++		ret = git_pathdup("fsmonitor--daemon.ipc");
     ++	return ret;
     ++}
     + \ No newline at end of file
     +
     + ## compat/fsmonitor/fsm-settings-darwin.c ##
     +@@
     + static enum fsmonitor_reason check_uds_volume(struct repository *r)
     + {
     + 	struct fs_info fs;
     +-	const char *ipc_path = fsmonitor_ipc__get_path();
     ++	const char *ipc_path = fsmonitor_ipc__get_path(r);
     + 	struct strbuf path = STRBUF_INIT;
     + 	strbuf_add(&path, ipc_path, strlen(ipc_path));
     + 
      
       ## contrib/buildsystems/CMakeLists.txt ##
      @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
     @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
       		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
      
       ## fsmonitor-ipc.c ##
     +@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
     + 	return 0;
     + }
     + 
     +-const char *fsmonitor_ipc__get_path(void)
     ++const char *fsmonitor_ipc__get_path(struct repository *r)
     + {
     + 	return NULL;
     + }
      @@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
       	return 1;
       }
     @@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
      -
       enum ipc_active_state fsmonitor_ipc__get_state(void)
       {
     - 	return ipc_get_active_state(fsmonitor_ipc__get_path());
     +-	return ipc_get_active_state(fsmonitor_ipc__get_path());
     ++	return ipc_get_active_state(fsmonitor_ipc__get_path(the_repository));
     + }
     + 
     + static int spawn_daemon(void)
     +@@ fsmonitor-ipc.c: int fsmonitor_ipc__send_query(const char *since_token,
     + 	trace2_data_string("fsm_client", NULL, "query/command", tok);
     + 
     + try_again:
     +-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
     +-				       &connection);
     ++	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
     ++						&options, &connection);
     + 
     + 	switch (state) {
     + 	case IPC_STATE__LISTENING:
     +@@ fsmonitor-ipc.c: try_again:
     + 
     + 	case IPC_STATE__INVALID_PATH:
     + 		ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
     +-			    fsmonitor_ipc__get_path());
     ++			    fsmonitor_ipc__get_path(the_repository));
     + 		goto done;
     + 
     + 	case IPC_STATE__OTHER_ERROR:
     + 	default:
     + 		ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
     +-			    fsmonitor_ipc__get_path());
     ++			    fsmonitor_ipc__get_path(the_repository));
     + 		goto done;
     + 	}
     + 
     +@@ fsmonitor-ipc.c: int fsmonitor_ipc__send_command(const char *command,
     + 	options.wait_if_busy = 1;
     + 	options.wait_if_not_found = 0;
     + 
     +-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
     +-				       &connection);
     ++	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
     ++						&options, &connection);
     + 	if (state != IPC_STATE__LISTENING) {
     + 		die(_("fsmonitor--daemon is not running"));
     + 		return -1;
     +
     + ## fsmonitor-ipc.h ##
     +@@
     + 
     + #include "simple-ipc.h"
     + 
     ++struct repository;
     ++
     + /*
     +  * Returns true if built-in file system monitor daemon is defined
     +  * for this platform.
     +@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
     +  *
     +  * Returns NULL if the daemon is not supported on this platform.
     +  */
     +-const char *fsmonitor_ipc__get_path(void);
     ++const char *fsmonitor_ipc__get_path(struct repository *r);
     + 
     + /*
     +  * Try to determine whether there is a `git-fsmonitor--daemon` process
 4:  3428bcf8763 = 3:  5518d2f3e03 fsmonitor: avoid socket location check if using hook
 5:  9c1f408ae6d ! 4:  3a9fe473cf4 fsmonitor: deal with synthetic firmlinks on macOS
     @@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
       };
       
       /*
     -@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
     -  */
     - void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
     - 
     -+char *fsmonitor_resolve_alias(const char *path,
     -+	struct alias_info *alias);
     -+
     - #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
     - #endif /* FSMONITOR_DAEMON_H */
      
       ## fsmonitor-path-utils.h ##
      @@
 -:  ----------- > 5:  4d00adb1deb fsmonitor: deal with synthetic firmlinks on macOS
 6:  d2c95e34d3a = 6:  260591f5820 fsmonitor: add documentation for allowRemote and socketDir options

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v7 1/6] fsmonitor: refactor filesystem checks to common interface
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
                               ` (6 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Provide a common interface for getting basic filesystem information
including filesystem type and whether the filesystem is remote.

Refactor existing code for getting basic filesystem info and detecting
remote file systems to the new interface.

Refactor filesystem checks to leverage new interface. For macOS,
error-out if the Unix Domain socket (UDS) file is on a remote
filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                                 |   1 +
 compat/fsmonitor/fsm-path-utils-darwin.c |  40 ++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 128 +++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  62 +++-----
 compat/fsmonitor/fsm-settings-win32.c    | 172 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   2 +
 fsmonitor-path-utils.h                   |  23 +++
 fsmonitor-settings.c                     |  50 +++++++
 8 files changed, 263 insertions(+), 215 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h

diff --git a/Makefile b/Makefile
index d9247ead45b..6e492a67547 100644
--- a/Makefile
+++ b/Makefile
@@ -2039,6 +2039,7 @@ endif
 ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
 ifeq ($(TCLTK_PATH),)
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
new file mode 100644
index 00000000000..067cbe6990a
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -0,0 +1,40 @@
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+#include <sys/param.h>
+#include <sys/mount.h>
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	struct statfs fs;
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return -1;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
+	if (!(fs.f_flags & MNT_LOCAL))
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	fs_info->typename = fs.f_fstypename;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
new file mode 100644
index 00000000000..a90b8f7925b
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -0,0 +1,128 @@
+#include "cache.h"
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Return -1 if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
+/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
+ *     (This is the normal method to access it.)
+ *
+ *     $ NET USE Z: \\server\share
+ *     $ git -C Z:/repo status
+ *
+ * (b) Windows allows a network share to be referenced WITHOUT mapping
+ *     it to drive letter.
+ *
+ *     $ NET USE \\server\share\dir
+ *     $ git -C //server/share/repo status
+ *
+ * (c) Windows allows "SUBST" to create a fake drive mapping to an
+ *     arbitrary path (which may be remote)
+ *
+ *     $ SUBST Q: Z:\repo
+ *     $ git -C Q:/ status
+ *
+ * (d) Windows allows a directory symlink to be created on a local
+ *     file system that points to a remote repo.
+ *
+ *     $ mklink /d ./link //server/share/repo
+ *     $ git -C ./link status
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	wchar_t wpath[MAX_PATH];
+	wchar_t wfullpath[MAX_PATH];
+	size_t wlen;
+	UINT driveType;
+
+	/*
+	 * Do everything in wide chars because the drive letter might be
+	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
+	 */
+	if (xutftowcs_path(wpath, path) < 0) {
+		return -1;
+	}
+
+	/*
+	 * GetDriveTypeW() requires a final slash.  We assume that the
+	 * worktree pathname points to an actual directory.
+	 */
+	wlen = wcslen(wpath);
+	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
+		wpath[wlen++] = L'\\';
+		wpath[wlen] = 0;
+	}
+
+	/*
+	 * Normalize the path.  If nothing else, this converts forward
+	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
+	 * correctly handle some UNC "\\server\share\..." paths.
+	 */
+	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) {
+		return -1;
+	}
+
+	driveType = GetDriveTypeW(wfullpath);
+	trace_printf_key(&trace_fsmonitor,
+			 "DriveType '%s' L'%ls' (%u)",
+			 path, wfullpath, driveType);
+
+	if (driveType == DRIVE_REMOTE) {
+		fs_info->is_remote = 1;
+		if (check_remote_protocol(wfullpath) < 0)
+			return -1;
+	} else {
+		fs_info->is_remote = 0;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dba3ced6bb7 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -1,32 +1,10 @@
-#include "cache.h"
 #include "config.h"
-#include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
-#include <sys/param.h>
-#include <sys/mount.h>
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
-/*
- * [1] Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type (NFS, SAMBA, etc.) dictates whether notification events
- * are available at all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
+ /*
  * For the builtin FSMonitor, we create the Unix domain socket for the
  * IPC in the .git directory.  If the working directory is remote,
  * then the socket will be created on the remote file system.  This
@@ -38,40 +16,34 @@
  * be taken to ensure that $HOME is actually local and not a managed
  * file share.)
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
+ * FAT32 and NTFS working directories are problematic too.
  *
  * The builtin FSMonitor uses a Unix domain socket in the .git
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
  */
-static enum fsmonitor_reason check_volume(struct repository *r)
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
-	struct statfs fs;
+	struct fs_info fs;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	struct strbuf path = STRBUF_INIT;
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
-	if (statfs(r->worktree, &fs) == -1) {
-		int saved_errno = errno;
-		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-				 r->worktree, strerror(saved_errno));
-		errno = saved_errno;
+	if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) {
+		strbuf_release(&path);
 		return FSMONITOR_REASON_ERROR;
 	}
 
-	trace_printf_key(&trace_fsmonitor,
-			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
 
-	if (!(fs.f_flags & MNT_LOCAL))
+	if (fs.is_remote)
 		return FSMONITOR_REASON_REMOTE;
 
-	if (!strcmp(fs.f_fstypename, "msdos")) /* aka FAT32 */
+	if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
 		return FSMONITOR_REASON_NOSOCKETS;
 
-	if (!strcmp(fs.f_fstypename, "ntfs"))
+	if (!strcmp(fs.typename, "ntfs"))
 		return FSMONITOR_REASON_NOSOCKETS;
 
 	return FSMONITOR_REASON_OK;
@@ -81,7 +53,7 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_volume(r);
+	reason = check_uds_volume(r);
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index e5ec5b0a9f7..d88b06ae610 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -1,8 +1,9 @@
 #include "cache.h"
 #include "config.h"
 #include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * VFS for Git is incompatible with FSMonitor.
@@ -24,171 +25,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
-/*
- * Check remote working directory protocol.
- *
- * Error if client machine cannot get remote protocol information.
- */
-static int check_remote_protocol(wchar_t *wpath)
-{
-	HANDLE h;
-	FILE_REMOTE_PROTOCOL_INFO proto_info;
-
-	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
-			FILE_FLAG_BACKUP_SEMANTICS, NULL);
-
-	if (h == INVALID_HANDLE_VALUE) {
-		error(_("[GLE %ld] unable to open for read '%ls'"),
-		      GetLastError(), wpath);
-		return -1;
-	}
-
-	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
-		&proto_info, sizeof(proto_info))) {
-		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
-		      GetLastError(), wpath);
-		CloseHandle(h);
-		return -1;
-	}
-
-	CloseHandle(h);
-
-	trace_printf_key(&trace_fsmonitor,
-				"check_remote_protocol('%ls') remote protocol %#8.8lx",
-				wpath, proto_info.Protocol);
-
-	return 0;
-}
-
-/*
- * Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type dictates whether notification events are available at
- * all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- * Notes for testing:
- *
- * (a) Windows allows a network share to be mapped to a drive letter.
- *     (This is the normal method to access it.)
- *
- *     $ NET USE Z: \\server\share
- *     $ git -C Z:/repo status
- *
- * (b) Windows allows a network share to be referenced WITHOUT mapping
- *     it to drive letter.
- *
- *     $ NET USE \\server\share\dir
- *     $ git -C //server/share/repo status
- *
- * (c) Windows allows "SUBST" to create a fake drive mapping to an
- *     arbitrary path (which may be remote)
- *
- *     $ SUBST Q: Z:\repo
- *     $ git -C Q:/ status
- *
- * (d) Windows allows a directory symlink to be created on a local
- *     file system that points to a remote repo.
- *
- *     $ mklink /d ./link //server/share/repo
- *     $ git -C ./link status
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
-	int ret;
-	wchar_t wpath[MAX_PATH];
-	wchar_t wfullpath[MAX_PATH];
-	size_t wlen;
-	UINT driveType;
-
-	/*
-	 * Do everything in wide chars because the drive letter might be
-	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
-	 */
-	if (xutftowcs_path(wpath, r->worktree) < 0)
-		return FSMONITOR_REASON_ERROR;
-
-	/*
-	 * GetDriveTypeW() requires a final slash.  We assume that the
-	 * worktree pathname points to an actual directory.
-	 */
-	wlen = wcslen(wpath);
-	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
-		wpath[wlen++] = L'\\';
-		wpath[wlen] = 0;
-	}
-
-	/*
-	 * Normalize the path.  If nothing else, this converts forward
-	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
-	 * correctly handle some UNC "\\server\share\..." paths.
-	 */
-	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
-		return FSMONITOR_REASON_ERROR;
-
-	driveType = GetDriveTypeW(wfullpath);
-	trace_printf_key(&trace_fsmonitor,
-			 "DriveType '%s' L'%ls' (%u)",
-			 r->worktree, wfullpath, driveType);
-
-	if (driveType == DRIVE_REMOTE) {
-		trace_printf_key(&trace_fsmonitor,
-				 "check_remote('%s') true",
-				 r->worktree);
-
-		ret = check_remote_protocol(wfullpath);
-		if (ret < 0)
-			return FSMONITOR_REASON_ERROR;
-
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
-	}
-
-	return FSMONITOR_REASON_OK;
-}
-
 enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
@@ -197,9 +33,5 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
-	reason = check_remote(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
-
 	return FSMONITOR_REASON_OK;
 }
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..b88494bf59b 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
new file mode 100644
index 00000000000..e48592887e7
--- /dev/null
+++ b/fsmonitor-path-utils.h
@@ -0,0 +1,23 @@
+#ifndef FSM_PATH_UTILS_H
+#define FSM_PATH_UTILS_H
+
+struct fs_info {
+	int is_remote;
+	char *typename;
+};
+
+/*
+ * Get some basic filesystem informtion for the given path
+ *
+ * Returns -1 on error, zero otherwise.
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+
+/*
+ * Determines if the filesystem that path resides on is remote.
+ *
+ * Returns -1 on error, 0 if not remote, 1 if remote.
+ */
+int fsmonitor__is_fs_remote(const char *path);
+
+#endif
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..d288cbad479 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 20:11               ` Junio C Hamano
  2022-09-16 19:53             ` [PATCH v7 3/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
                               ` (5 subsequent siblings)
  7 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If the .git directory is on a remote file system, create the socket
file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                               |  1 +
 builtin/fsmonitor--daemon.c            |  3 +-
 compat/fsmonitor/fsm-ipc-darwin.c      | 49 ++++++++++++++++++++++++++
 compat/fsmonitor/fsm-ipc-win32.c       |  9 +++++
 compat/fsmonitor/fsm-settings-darwin.c |  2 +-
 contrib/buildsystems/CMakeLists.txt    |  2 ++
 fsmonitor-ipc.c                        | 18 +++++-----
 fsmonitor-ipc.h                        |  4 ++-
 8 files changed, 75 insertions(+), 13 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c

diff --git a/Makefile b/Makefile
index 6e492a67547..58bb9248471 100644
--- a/Makefile
+++ b/Makefile
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
 endif
 
 ifdef FSMONITOR_OS_SETTINGS
diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 2c109cf8b37..0123fc33ed2 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)
 	 * directory.)
 	 */
 	strbuf_init(&state.path_ipc, 0);
-	strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path()));
+	strbuf_addstr(&state.path_ipc,
+		absolute_path(fsmonitor_ipc__get_path(the_repository)));
 
 	/*
 	 * Confirm that we can create platform-specific resources for the
diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
new file mode 100644
index 00000000000..c238ed9c5d7
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-darwin.c
@@ -0,0 +1,49 @@
+#include "cache.h"
+#include "config.h"
+#include "strbuf.h"
+#include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(struct repository *r)
+{
+	static const char *ipc_path;
+	SHA_CTX sha1ctx;
+	char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (ipc_path)
+		return ipc_path;
+
+	if (!r)
+		r = the_repository;
+
+	ipc_path = fsmonitor_ipc__get_default_path();
+
+	/* By default the socket file is created in the .git directory */
+	if (fsmonitor__is_fs_remote(ipc_path) < 1)
+		return ipc_path;
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
+	SHA1_Final(hash, &sha1ctx);
+
+	repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+	/* Create the socket file in either socketDir or $HOME */
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+
+	ipc_path = interpolate_path(ipc_file.buf, 1);
+	if (!ipc_path)
+		die(_("Invalid path: %s"), ipc_file.buf);
+
+	strbuf_release(&ipc_file);
+	return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
new file mode 100644
index 00000000000..3a3a46db209
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-win32.c
@@ -0,0 +1,9 @@
+#include "config.h"
+#include "fsmonitor-ipc.h"
+
+const char *fsmonitor_ipc__get_path(struct repository *r) {
+	static char *ret;
+	if (!ret)
+		ret = git_pathdup("fsmonitor--daemon.ipc");
+	return ret;
+}
\ No newline at end of file
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index dba3ced6bb7..681d8bf963e 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -26,7 +26,7 @@
 static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
 	struct fs_info fs;
-	const char *ipc_path = fsmonitor_ipc__get_path();
+	const char *ipc_path = fsmonitor_ipc__get_path(r);
 	struct strbuf path = STRBUF_INIT;
 	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index b88494bf59b..7e7b6b9a362 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..c0f42301c84 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)
 	return 0;
 }
 
-const char *fsmonitor_ipc__get_path(void)
+const char *fsmonitor_ipc__get_path(struct repository *r)
 {
 	return NULL;
 }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
-	return ipc_get_active_state(fsmonitor_ipc__get_path());
+	return ipc_get_active_state(fsmonitor_ipc__get_path(the_repository));
 }
 
 static int spawn_daemon(void)
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,
 	trace2_data_string("fsm_client", NULL, "query/command", tok);
 
 try_again:
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 
 	switch (state) {
 	case IPC_STATE__LISTENING:
@@ -117,13 +115,13 @@ try_again:
 
 	case IPC_STATE__INVALID_PATH:
 		ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 
 	case IPC_STATE__OTHER_ERROR:
 	default:
 		ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 	}
 
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command,
 	options.wait_if_busy = 1;
 	options.wait_if_not_found = 0;
 
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 	if (state != IPC_STATE__LISTENING) {
 		die(_("fsmonitor--daemon is not running"));
 		return -1;
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..8b489da762b 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v7 3/6] fsmonitor: avoid socket location check if using hook
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 4/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
                               ` (4 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If monitoring is done via fsmonitor hook rather than IPC there is no
need to check if the location of the Unix Domain socket (UDS) file is
on a remote filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
 compat/fsmonitor/fsm-settings-win32.c  |  2 +-
 fsmonitor-settings.c                   |  8 ++++----
 fsmonitor-settings.h                   |  2 +-
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index 681d8bf963e..40da2d3b533 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_uds_volume(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
+	if (ipc) {
+		reason = check_uds_volume(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
+	}
 
 	return FSMONITOR_REASON_OK;
 }
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index d88b06ae610..a8af31b71de 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index d288cbad479..531a1b6f956 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 }
 #endif
 
-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
 {
 	if (!r->worktree) {
 		/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 		reason = check_remote(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-		reason = fsm_os__incompatible(r);
+		reason = fsm_os__incompatible(r, ipc);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
 	}
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..0721617b95a 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v7 4/6] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
                               ` (2 preceding siblings ...)
  2022-09-16 19:53             ` [PATCH v7 3/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 19:53             ` [PATCH v7 5/6] " Eric DeCosta via GitGitGadget
                               ` (3 subsequent siblings)
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     |  6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  3 +
 fsmonitor-path-utils.h                   | 36 ++++++++++
 6 files changed, 161 insertions(+), 1 deletion(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 0123fc33ed2..56fcd1c2baa 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	state.alias.alias = NULL;
+	state.alias.points_to = NULL;
+	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+		err = error(_("could not get worktree alias"));
+		goto done;
+	}
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..179886bc15b 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (!path_k)
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index 067cbe6990a..13807f58e95 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -38,3 +41,92 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR * dir;
+	int read;
+	int retval;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to;
+
+	retval = 0;
+	dir = opendir("/");
+	if (!dir)
+		return -1;
+
+	strbuf_init(&alias, 256);
+	strbuf_init(&points_to, MAXPATHLEN);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addch(&alias, '/');
+		strbuf_add(&alias, de->d_name, strlen(de->d_name));
+
+		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
+		if (read > 0) {
+			strbuf_setlen(&points_to, read);
+			if ((strncmp(points_to.buf, path, points_to.len) == 0)
+				&& path[points_to.len] == '/') {
+				info->alias = strbuf_detach(&alias, NULL);
+				info->points_to = strbuf_detach(&points_to, NULL);
+				trace_printf_key(&trace_fsmonitor,
+					"Found alias for '%s' : '%s' -> '%s'",
+					path, info->alias, info->points_to);
+				retval = 0;
+				goto done;
+			}
+		} else if (errno != EINVAL) { /* Something other than not a link */
+			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
+			retval = -1;
+			goto done;
+		}
+	}
+
+	done:
+	closedir(dir);
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	int len = info->alias ? strlen(info->alias) : 0;
+
+	if (!len)
+		return NULL;
+
+	if ((strncmp(info->alias, path, len) == 0)
+		&& path[len] == '/') {
+		struct strbuf tmp;
+		const char *remainder = path + len;
+		int ptr_len = strlen(info->points_to);
+		int rem_len = strlen(remainder);
+
+		strbuf_init(&tmp, ptr_len + rem_len);
+		strbuf_add(&tmp, info->points_to, ptr_len);
+		strbuf_add(&tmp, remainder, rem_len);
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index a90b8f7925b..0d95bbb416f 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..e24838f9a86 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index e48592887e7..50ef37e57bb 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v7 5/6] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
                               ` (3 preceding siblings ...)
  2022-09-16 19:53             ` [PATCH v7 4/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 20:15               ` Junio C Hamano
  2022-09-16 19:53             ` [PATCH v7 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
                               ` (2 subsequent siblings)
  7 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor--daemon.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index e24838f9a86..98cbb430083 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -170,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,
  */
 void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
 
+char *fsmonitor_resolve_alias(const char *path,
+	struct alias_info *alias);
+
 #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
 #endif /* FSMONITOR_DAEMON_H */
-- 
@@ -170,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,
  */
 void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
 
+char *fsmonitor_resolve_alias(const char *path,
+	struct alias_info *alias);
+
 #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
 #endif /* FSMONITOR_DAEMON_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v7 6/6] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
                               ` (4 preceding siblings ...)
  2022-09-16 19:53             ` [PATCH v7 5/6] " Eric DeCosta via GitGitGadget
@ 2022-09-16 19:53             ` Eric DeCosta via GitGitGadget
  2022-09-16 20:09             ` [PATCH v7 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
  7 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-16 19:53 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
Call-out experimental nature of 'fsmonitor.allowRemote' and limited file
system support for 'fsmonitor.socketDir'.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt
index cc142fb8612..0adccd0eced 100644
--- a/Documentation/git-fsmonitor--daemon.txt
+++ b/Documentation/git-fsmonitor--daemon.txt
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* RE: [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-16 17:58           ` [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Jeff Hostetler
@ 2022-09-16 20:02             ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-16 20:02 UTC (permalink / raw)
  To: Jeff Hostetler, Eric DeCosta via GitGitGadget, git
  Cc: Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Friday, September 16, 2022 1:59 PM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org
> Cc: Eric Sunshine <sunshine@sunshineco.com>; Torsten Bögershausen
> <tboegi@web.de>; Ævar Arnfjörð Bjarmason <avarab@gmail.com>; Ramsay
> Jones <ramsay@ramsayjones.plus.com>; Johannes Schindelin
> <Johannes.Schindelin@gmx.de>; Eric DeCosta <edecosta@mathworks.com>
> Subject: Re: [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run
> against network-mounted repos
> 
> 
> 
> On 9/13/22 4:27 PM, Eric DeCosta via GitGitGadget wrote:
> > Follow-on to the work done to allow Windows to work against
> > network-mounted repos for macOS.
> >
> > Have macOS take advantage of the same configuration option,
> > 'fsmonitor.allowRemote' that was introduced for Windows. Setting this
> > option to true will override the default behavior (erroring-out) when
> > a network-mounted repo is detected by fsmonitor.
> >
> > The added wrinkle being that the Unix domain socket (UDS) file used
> > for IPC cannot be created in a network location; instead $HOME is used
> > if the default location is on the network. The user may, optionally,
> > set the 'fsmonitor.socketDir' configuration option to a valid, local
> > directory if $HOME itself is on the network or is simply not the
> > desired location for the UDS file.
> >
> > An additional issue is that for mount points in the root directory,
> > FSEvents does not report a path that matches the worktree directory
> > due to the introduction of 'synthetic firmlinks'. fsmonitor must map
> > the FSEvents paths to the worktree directory by interrogating the root
> > filesystem for synthetic firmlinks and using that information to translate the
> path.
> >
> > v6 differs from v5:
> >
> >   * incorporates earlier, Windows-specific changes that have not made it
> back
> >     yet to the master branch
> >   * incorporates code review feedback
> >   * adds documentation for 'fsmonitor.allowRemote' and
> 'fsmonitor.socketDir'
> >
> > v5 differs significantly from earlier versions:
> >
> >   * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
> >     such that these options are no longer added to the settings data
> >     structure but are rather read from config at point of use
> >   * refactoring of code for handling platform-specific file system checks via
> >     a common interface to avoid platform #ifdef in IPC code and be in-model
> >     with other platform-specific fsmonitor code
> >   * dealing with 'synthetic firmlinks' on macOS
> >
> 
> I've looked at v5 and v6 and I like the direction this is heading, so I'll mark this
> LGTM.  (I'm still traveling back from Git Merge, so I haven't had time to test it
> out, but I think we should proceed with it.)
> 
> Thanks for you patience and attention to detail on this!
> Jeff

Great! I just created v7 with some small updates based on Junio's feedback.

I think this landed in a pretty good place. I don't know that the plans are for fsmonitor on Linux, but I've already started tinkering around with an inotify-based implementation for that platform. The devil is in the details, of course, but it looks fairly straight-forward.

-Eric

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
                               ` (5 preceding siblings ...)
  2022-09-16 19:53             ` [PATCH v7 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
@ 2022-09-16 20:09             ` Junio C Hamano
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
  7 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-09-16 20:09 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

>  5:  9c1f408ae6d ! 4:  3a9fe473cf4 fsmonitor: deal with synthetic firmlinks on macOS
>      @@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
>        };
>        
>        /*
>      -@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
>      -  */
>      - void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
>      - 
>      -+char *fsmonitor_resolve_alias(const char *path,
>      -+	struct alias_info *alias);
>      -+
>      - #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
>      - #endif /* FSMONITOR_DAEMON_H */
>       
>        ## fsmonitor-path-utils.h ##
>       @@
>  -:  ----------- > 5:  4d00adb1deb fsmonitor: deal with synthetic firmlinks on macOS

"rebase -i" mistake or something?

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-16 19:53             ` [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-16 20:11               ` Junio C Hamano
  2022-09-19 12:31                 ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-16 20:11 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> +const char *fsmonitor_ipc__get_path(struct repository *r)
> +{
> +	static const char *ipc_path;
> +	SHA_CTX sha1ctx;
> +	char *sock_dir;
> +	struct strbuf ipc_file = STRBUF_INIT;
> +	unsigned char hash[SHA_DIGEST_LENGTH];
> +
> +	if (ipc_path)
> +		return ipc_path;
> +
> +	if (!r)
> +		r = the_repository;

I'd prefer not to see this "NULL means the_repository".  It would be
a different story if the caller does not necessarily have a ready
access to the_repository, but it is a global, so the caller can pass
the_repository and be more explicit.  Giving two ways to the caller
to express same thing is not a good idea.

Thanks.


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 5/6] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-16 19:53             ` [PATCH v7 5/6] " Eric DeCosta via GitGitGadget
@ 2022-09-16 20:15               ` Junio C Hamano
  0 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-09-16 20:15 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: git, Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

"Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Starting with macOS 10.15 (Catalina), Apple introduced a new feature
> called 'firmlinks' in order to separate the boot volume into two
> volumes, one read-only and one writable but still present them to the
> user as a single volume. Along with this change, Apple removed the
> ...
> Therefore, in order to deal with synthetic firmlinks, the root directory
> is scanned and the first possible synthetic firmink that, when resolved,
> is a prefix of the worktree is used to map FSEvents paths to worktree
> paths.

The proposed log message for this step seems fairly similar to the
previous one.

> diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
> index e24838f9a86..98cbb430083 100644
> --- a/fsmonitor--daemon.h
> +++ b/fsmonitor--daemon.h
> @@ -170,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,
>   */
>  void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
>  
> +char *fsmonitor_resolve_alias(const char *path,
> +	struct alias_info *alias);
> +

After applying all the 6 steps, this function is declared here
without any implementation.  Should we drop this step, I have to
wonder.

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v8 0/5] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-16 19:53           ` [PATCH v7 " Eric DeCosta via GitGitGadget
                               ` (6 preceding siblings ...)
  2022-09-16 20:09             ` [PATCH v7 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Junio C Hamano
@ 2022-09-17  1:12             ` Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 1/5] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                                 ` (5 more replies)
  7 siblings, 6 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

v8 differs from v7:

 * incorporates code review feedback
 * gets the rebase right

v7 differs from v6:

 * incorporates code review feedback

v6 differs from v5:

 * incorporates earlier, Windows-specific changes that have not made it back
   yet to the master branch
 * incorporates code review feedback
 * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'

v5 differs significantly from earlier versions:

 * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
   such that these options are no longer added to the settings data
   structure but are rather read from config at point of use
 * refactoring of code for handling platform-specific file system checks via
   a common interface to avoid platform #ifdef in IPC code and be in-model
   with other platform-specific fsmonitor code
 * dealing with 'synthetic firmlinks' on macOS

Eric DeCosta (5):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: add documentation for allowRemote and socketDir options

 Documentation/git-fsmonitor--daemon.txt  |  35 +++++
 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |  11 +-
 compat/fsmonitor/fsm-ipc-darwin.c        |  46 ++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   9 ++
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 145 +++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  70 +++------
 compat/fsmonitor/fsm-settings-win32.c    | 174 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   3 +
 fsmonitor-ipc.c                          |  18 ++-
 fsmonitor-ipc.h                          |   4 +-
 fsmonitor-path-utils.h                   |  59 ++++++++
 fsmonitor-settings.c                     |  58 +++++++-
 fsmonitor-settings.h                     |   2 +-
 17 files changed, 541 insertions(+), 237 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v8
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v8
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v7:

 1:  155a6890806 = 1:  155a6890806 fsmonitor: refactor filesystem checks to common interface
 2:  075340bd2a7 ! 2:  b5356497228 fsmonitor: relocate socket file if .git directory is remote
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +	if (ipc_path)
      +		return ipc_path;
      +
     -+	if (!r)
     -+		r = the_repository;
     -+
      +	ipc_path = fsmonitor_ipc__get_default_path();
      +
      +	/* By default the socket file is created in the .git directory */
 3:  5518d2f3e03 = 3:  6719ca2b24d fsmonitor: avoid socket location check if using hook
 4:  3a9fe473cf4 ! 4:  d736cb8fa90 fsmonitor: deal with synthetic firmlinks on macOS
     @@ Commit message
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     +    fsmonitor: deal with synthetic firmlinks on macOS
     +
     +    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
     +    called 'firmlinks' in order to separate the boot volume into two
     +    volumes, one read-only and one writable but still present them to the
     +    user as a single volume. Along with this change, Apple removed the
     +    ability to create symlinks in the root directory and replaced them with
     +    'synthetic firmlinks'. See 'man synthetic.conf'
     +
     +    When FSEevents reports the path of changed files, if the path involves
     +    a synthetic firmlink, the path is reported from the point of the
     +    synthetic firmlink and not the real path. For example:
     +
     +    Real path:
     +    /System/Volumes/Data/network/working/directory/foo.txt
     +
     +    Synthetic firmlink:
     +    /network -> /System/Volumes/Data/network
     +
     +    FSEvents path:
     +    /network/working/directory/foo.txt
     +
     +    This causes the FSEvents path to not match against the worktree
     +    directory.
     +
     +    There are several ways in which synthetic firmlinks can be created:
     +    they can be defined in /etc/synthetic.conf, the automounter can create
     +    them, and there may be other means. Simply reading /etc/synthetic.conf
     +    is insufficient. No matter what process creates synthetic firmlinks,
     +    they all get created in the root directory.
     +
     +    Therefore, in order to deal with synthetic firmlinks, the root directory
     +    is scanned and the first possible synthetic firmink that, when resolved,
     +    is a prefix of the worktree is used to map FSEvents paths to worktree
     +    paths.
     +
     +    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
     +
     +    fsmonitor: deal with synthetic firmlinks on macOS
     +
     +    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
     +    called 'firmlinks' in order to separate the boot volume into two
     +    volumes, one read-only and one writable but still present them to the
     +    user as a single volume. Along with this change, Apple removed the
     +    ability to create symlinks in the root directory and replaced them with
     +    'synthetic firmlinks'. See 'man synthetic.conf'
     +
     +    When FSEevents reports the path of changed files, if the path involves
     +    a synthetic firmlink, the path is reported from the point of the
     +    synthetic firmlink and not the real path. For example:
     +
     +    Real path:
     +    /System/Volumes/Data/network/working/directory/foo.txt
     +
     +    Synthetic firmlink:
     +    /network -> /System/Volumes/Data/network
     +
     +    FSEvents path:
     +    /network/working/directory/foo.txt
     +
     +    This causes the FSEvents path to not match against the worktree
     +    directory.
     +
     +    There are several ways in which synthetic firmlinks can be created:
     +    they can be defined in /etc/synthetic.conf, the automounter can create
     +    them, and there may be other means. Simply reading /etc/synthetic.conf
     +    is insufficient. No matter what process creates synthetic firmlinks,
     +    they all get created in the root directory.
     +
     +    Therefore, in order to deal with synthetic firmlinks, the root directory
     +    is scanned and the first possible synthetic firmink that, when resolved,
     +    is a prefix of the worktree is used to map FSEvents paths to worktree
     +    paths.
     +
     +    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
     +
       ## builtin/fsmonitor--daemon.c ##
      @@
       #include "parse-options.h"
 5:  4d00adb1deb < -:  ----------- fsmonitor: deal with synthetic firmlinks on macOS
 6:  260591f5820 = 5:  ddf4e3e6442 fsmonitor: add documentation for allowRemote and socketDir options

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v8 1/5] fsmonitor: refactor filesystem checks to common interface
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
@ 2022-09-17  1:12               ` Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
                                 ` (4 subsequent siblings)
  5 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Provide a common interface for getting basic filesystem information
including filesystem type and whether the filesystem is remote.

Refactor existing code for getting basic filesystem info and detecting
remote file systems to the new interface.

Refactor filesystem checks to leverage new interface. For macOS,
error-out if the Unix Domain socket (UDS) file is on a remote
filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                                 |   1 +
 compat/fsmonitor/fsm-path-utils-darwin.c |  40 ++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 128 +++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  62 +++-----
 compat/fsmonitor/fsm-settings-win32.c    | 172 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   2 +
 fsmonitor-path-utils.h                   |  23 +++
 fsmonitor-settings.c                     |  50 +++++++
 8 files changed, 263 insertions(+), 215 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h

diff --git a/Makefile b/Makefile
index d9247ead45b..6e492a67547 100644
--- a/Makefile
+++ b/Makefile
@@ -2039,6 +2039,7 @@ endif
 ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
 ifeq ($(TCLTK_PATH),)
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
new file mode 100644
index 00000000000..067cbe6990a
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -0,0 +1,40 @@
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+#include <sys/param.h>
+#include <sys/mount.h>
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	struct statfs fs;
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return -1;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
+	if (!(fs.f_flags & MNT_LOCAL))
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	fs_info->typename = fs.f_fstypename;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
new file mode 100644
index 00000000000..a90b8f7925b
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -0,0 +1,128 @@
+#include "cache.h"
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Return -1 if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
+/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
+ *     (This is the normal method to access it.)
+ *
+ *     $ NET USE Z: \\server\share
+ *     $ git -C Z:/repo status
+ *
+ * (b) Windows allows a network share to be referenced WITHOUT mapping
+ *     it to drive letter.
+ *
+ *     $ NET USE \\server\share\dir
+ *     $ git -C //server/share/repo status
+ *
+ * (c) Windows allows "SUBST" to create a fake drive mapping to an
+ *     arbitrary path (which may be remote)
+ *
+ *     $ SUBST Q: Z:\repo
+ *     $ git -C Q:/ status
+ *
+ * (d) Windows allows a directory symlink to be created on a local
+ *     file system that points to a remote repo.
+ *
+ *     $ mklink /d ./link //server/share/repo
+ *     $ git -C ./link status
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	wchar_t wpath[MAX_PATH];
+	wchar_t wfullpath[MAX_PATH];
+	size_t wlen;
+	UINT driveType;
+
+	/*
+	 * Do everything in wide chars because the drive letter might be
+	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
+	 */
+	if (xutftowcs_path(wpath, path) < 0) {
+		return -1;
+	}
+
+	/*
+	 * GetDriveTypeW() requires a final slash.  We assume that the
+	 * worktree pathname points to an actual directory.
+	 */
+	wlen = wcslen(wpath);
+	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
+		wpath[wlen++] = L'\\';
+		wpath[wlen] = 0;
+	}
+
+	/*
+	 * Normalize the path.  If nothing else, this converts forward
+	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
+	 * correctly handle some UNC "\\server\share\..." paths.
+	 */
+	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) {
+		return -1;
+	}
+
+	driveType = GetDriveTypeW(wfullpath);
+	trace_printf_key(&trace_fsmonitor,
+			 "DriveType '%s' L'%ls' (%u)",
+			 path, wfullpath, driveType);
+
+	if (driveType == DRIVE_REMOTE) {
+		fs_info->is_remote = 1;
+		if (check_remote_protocol(wfullpath) < 0)
+			return -1;
+	} else {
+		fs_info->is_remote = 0;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dba3ced6bb7 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -1,32 +1,10 @@
-#include "cache.h"
 #include "config.h"
-#include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
-#include <sys/param.h>
-#include <sys/mount.h>
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
-/*
- * [1] Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type (NFS, SAMBA, etc.) dictates whether notification events
- * are available at all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
+ /*
  * For the builtin FSMonitor, we create the Unix domain socket for the
  * IPC in the .git directory.  If the working directory is remote,
  * then the socket will be created on the remote file system.  This
@@ -38,40 +16,34 @@
  * be taken to ensure that $HOME is actually local and not a managed
  * file share.)
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
+ * FAT32 and NTFS working directories are problematic too.
  *
  * The builtin FSMonitor uses a Unix domain socket in the .git
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
  */
-static enum fsmonitor_reason check_volume(struct repository *r)
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
-	struct statfs fs;
+	struct fs_info fs;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	struct strbuf path = STRBUF_INIT;
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
-	if (statfs(r->worktree, &fs) == -1) {
-		int saved_errno = errno;
-		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-				 r->worktree, strerror(saved_errno));
-		errno = saved_errno;
+	if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) {
+		strbuf_release(&path);
 		return FSMONITOR_REASON_ERROR;
 	}
 
-	trace_printf_key(&trace_fsmonitor,
-			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
 
-	if (!(fs.f_flags & MNT_LOCAL))
+	if (fs.is_remote)
 		return FSMONITOR_REASON_REMOTE;
 
-	if (!strcmp(fs.f_fstypename, "msdos")) /* aka FAT32 */
+	if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
 		return FSMONITOR_REASON_NOSOCKETS;
 
-	if (!strcmp(fs.f_fstypename, "ntfs"))
+	if (!strcmp(fs.typename, "ntfs"))
 		return FSMONITOR_REASON_NOSOCKETS;
 
 	return FSMONITOR_REASON_OK;
@@ -81,7 +53,7 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_volume(r);
+	reason = check_uds_volume(r);
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index e5ec5b0a9f7..d88b06ae610 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -1,8 +1,9 @@
 #include "cache.h"
 #include "config.h"
 #include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * VFS for Git is incompatible with FSMonitor.
@@ -24,171 +25,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
-/*
- * Check remote working directory protocol.
- *
- * Error if client machine cannot get remote protocol information.
- */
-static int check_remote_protocol(wchar_t *wpath)
-{
-	HANDLE h;
-	FILE_REMOTE_PROTOCOL_INFO proto_info;
-
-	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
-			FILE_FLAG_BACKUP_SEMANTICS, NULL);
-
-	if (h == INVALID_HANDLE_VALUE) {
-		error(_("[GLE %ld] unable to open for read '%ls'"),
-		      GetLastError(), wpath);
-		return -1;
-	}
-
-	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
-		&proto_info, sizeof(proto_info))) {
-		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
-		      GetLastError(), wpath);
-		CloseHandle(h);
-		return -1;
-	}
-
-	CloseHandle(h);
-
-	trace_printf_key(&trace_fsmonitor,
-				"check_remote_protocol('%ls') remote protocol %#8.8lx",
-				wpath, proto_info.Protocol);
-
-	return 0;
-}
-
-/*
- * Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type dictates whether notification events are available at
- * all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- * Notes for testing:
- *
- * (a) Windows allows a network share to be mapped to a drive letter.
- *     (This is the normal method to access it.)
- *
- *     $ NET USE Z: \\server\share
- *     $ git -C Z:/repo status
- *
- * (b) Windows allows a network share to be referenced WITHOUT mapping
- *     it to drive letter.
- *
- *     $ NET USE \\server\share\dir
- *     $ git -C //server/share/repo status
- *
- * (c) Windows allows "SUBST" to create a fake drive mapping to an
- *     arbitrary path (which may be remote)
- *
- *     $ SUBST Q: Z:\repo
- *     $ git -C Q:/ status
- *
- * (d) Windows allows a directory symlink to be created on a local
- *     file system that points to a remote repo.
- *
- *     $ mklink /d ./link //server/share/repo
- *     $ git -C ./link status
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
-	int ret;
-	wchar_t wpath[MAX_PATH];
-	wchar_t wfullpath[MAX_PATH];
-	size_t wlen;
-	UINT driveType;
-
-	/*
-	 * Do everything in wide chars because the drive letter might be
-	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
-	 */
-	if (xutftowcs_path(wpath, r->worktree) < 0)
-		return FSMONITOR_REASON_ERROR;
-
-	/*
-	 * GetDriveTypeW() requires a final slash.  We assume that the
-	 * worktree pathname points to an actual directory.
-	 */
-	wlen = wcslen(wpath);
-	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
-		wpath[wlen++] = L'\\';
-		wpath[wlen] = 0;
-	}
-
-	/*
-	 * Normalize the path.  If nothing else, this converts forward
-	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
-	 * correctly handle some UNC "\\server\share\..." paths.
-	 */
-	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
-		return FSMONITOR_REASON_ERROR;
-
-	driveType = GetDriveTypeW(wfullpath);
-	trace_printf_key(&trace_fsmonitor,
-			 "DriveType '%s' L'%ls' (%u)",
-			 r->worktree, wfullpath, driveType);
-
-	if (driveType == DRIVE_REMOTE) {
-		trace_printf_key(&trace_fsmonitor,
-				 "check_remote('%s') true",
-				 r->worktree);
-
-		ret = check_remote_protocol(wfullpath);
-		if (ret < 0)
-			return FSMONITOR_REASON_ERROR;
-
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
-	}
-
-	return FSMONITOR_REASON_OK;
-}
-
 enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
@@ -197,9 +33,5 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
-	reason = check_remote(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
-
 	return FSMONITOR_REASON_OK;
 }
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..b88494bf59b 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
new file mode 100644
index 00000000000..e48592887e7
--- /dev/null
+++ b/fsmonitor-path-utils.h
@@ -0,0 +1,23 @@
+#ifndef FSM_PATH_UTILS_H
+#define FSM_PATH_UTILS_H
+
+struct fs_info {
+	int is_remote;
+	char *typename;
+};
+
+/*
+ * Get some basic filesystem informtion for the given path
+ *
+ * Returns -1 on error, zero otherwise.
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+
+/*
+ * Determines if the filesystem that path resides on is remote.
+ *
+ * Returns -1 on error, 0 if not remote, 1 if remote.
+ */
+int fsmonitor__is_fs_remote(const char *path);
+
+#endif
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..d288cbad479 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 1/5] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
@ 2022-09-17  1:12               ` Eric DeCosta via GitGitGadget
  2022-09-17  4:13                 ` Eric Sunshine
  2022-09-17  6:29                 ` Eric Sunshine
  2022-09-17  1:12               ` [PATCH v8 3/5] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
                                 ` (3 subsequent siblings)
  5 siblings, 2 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If the .git directory is on a remote file system, create the socket
file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                               |  1 +
 builtin/fsmonitor--daemon.c            |  3 +-
 compat/fsmonitor/fsm-ipc-darwin.c      | 46 ++++++++++++++++++++++++++
 compat/fsmonitor/fsm-ipc-win32.c       |  9 +++++
 compat/fsmonitor/fsm-settings-darwin.c |  2 +-
 contrib/buildsystems/CMakeLists.txt    |  2 ++
 fsmonitor-ipc.c                        | 18 +++++-----
 fsmonitor-ipc.h                        |  4 ++-
 8 files changed, 72 insertions(+), 13 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c

diff --git a/Makefile b/Makefile
index 6e492a67547..58bb9248471 100644
--- a/Makefile
+++ b/Makefile
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
 endif
 
 ifdef FSMONITOR_OS_SETTINGS
diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 2c109cf8b37..0123fc33ed2 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)
 	 * directory.)
 	 */
 	strbuf_init(&state.path_ipc, 0);
-	strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path()));
+	strbuf_addstr(&state.path_ipc,
+		absolute_path(fsmonitor_ipc__get_path(the_repository)));
 
 	/*
 	 * Confirm that we can create platform-specific resources for the
diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
new file mode 100644
index 00000000000..d6628185000
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-darwin.c
@@ -0,0 +1,46 @@
+#include "cache.h"
+#include "config.h"
+#include "strbuf.h"
+#include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(struct repository *r)
+{
+	static const char *ipc_path;
+	SHA_CTX sha1ctx;
+	char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (ipc_path)
+		return ipc_path;
+
+	ipc_path = fsmonitor_ipc__get_default_path();
+
+	/* By default the socket file is created in the .git directory */
+	if (fsmonitor__is_fs_remote(ipc_path) < 1)
+		return ipc_path;
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
+	SHA1_Final(hash, &sha1ctx);
+
+	repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+	/* Create the socket file in either socketDir or $HOME */
+	if (sock_dir && *sock_dir)
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+	else
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+
+	ipc_path = interpolate_path(ipc_file.buf, 1);
+	if (!ipc_path)
+		die(_("Invalid path: %s"), ipc_file.buf);
+
+	strbuf_release(&ipc_file);
+	return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
new file mode 100644
index 00000000000..3a3a46db209
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-win32.c
@@ -0,0 +1,9 @@
+#include "config.h"
+#include "fsmonitor-ipc.h"
+
+const char *fsmonitor_ipc__get_path(struct repository *r) {
+	static char *ret;
+	if (!ret)
+		ret = git_pathdup("fsmonitor--daemon.ipc");
+	return ret;
+}
\ No newline at end of file
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index dba3ced6bb7..681d8bf963e 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -26,7 +26,7 @@
 static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
 	struct fs_info fs;
-	const char *ipc_path = fsmonitor_ipc__get_path();
+	const char *ipc_path = fsmonitor_ipc__get_path(r);
 	struct strbuf path = STRBUF_INIT;
 	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index b88494bf59b..7e7b6b9a362 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..c0f42301c84 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)
 	return 0;
 }
 
-const char *fsmonitor_ipc__get_path(void)
+const char *fsmonitor_ipc__get_path(struct repository *r)
 {
 	return NULL;
 }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
-	return ipc_get_active_state(fsmonitor_ipc__get_path());
+	return ipc_get_active_state(fsmonitor_ipc__get_path(the_repository));
 }
 
 static int spawn_daemon(void)
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,
 	trace2_data_string("fsm_client", NULL, "query/command", tok);
 
 try_again:
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 
 	switch (state) {
 	case IPC_STATE__LISTENING:
@@ -117,13 +115,13 @@ try_again:
 
 	case IPC_STATE__INVALID_PATH:
 		ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 
 	case IPC_STATE__OTHER_ERROR:
 	default:
 		ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 	}
 
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command,
 	options.wait_if_busy = 1;
 	options.wait_if_not_found = 0;
 
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 	if (state != IPC_STATE__LISTENING) {
 		die(_("fsmonitor--daemon is not running"));
 		return -1;
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..8b489da762b 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v8 3/5] fsmonitor: avoid socket location check if using hook
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 1/5] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-17  1:12               ` Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 4/5] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
                                 ` (2 subsequent siblings)
  5 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If monitoring is done via fsmonitor hook rather than IPC there is no
need to check if the location of the Unix Domain socket (UDS) file is
on a remote filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
 compat/fsmonitor/fsm-settings-win32.c  |  2 +-
 fsmonitor-settings.c                   |  8 ++++----
 fsmonitor-settings.h                   |  2 +-
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index 681d8bf963e..40da2d3b533 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_uds_volume(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
+	if (ipc) {
+		reason = check_uds_volume(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
+	}
 
 	return FSMONITOR_REASON_OK;
 }
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index d88b06ae610..a8af31b71de 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index d288cbad479..531a1b6f956 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 }
 #endif
 
-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
 {
 	if (!r->worktree) {
 		/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 		reason = check_remote(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-		reason = fsm_os__incompatible(r);
+		reason = fsm_os__incompatible(r, ipc);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
 	}
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..0721617b95a 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v8 4/5] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
                                 ` (2 preceding siblings ...)
  2022-09-17  1:12               ` [PATCH v8 3/5] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-17  1:12               ` Eric DeCosta via GitGitGadget
  2022-09-17  1:12               ` [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  5 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>

fsmonitor: deal with synthetic firmlinks on macOS

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>

fsmonitor: deal with synthetic firmlinks on macOS

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     |  6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  3 +
 fsmonitor-path-utils.h                   | 36 ++++++++++
 6 files changed, 161 insertions(+), 1 deletion(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 0123fc33ed2..56fcd1c2baa 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	state.alias.alias = NULL;
+	state.alias.points_to = NULL;
+	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+		err = error(_("could not get worktree alias"));
+		goto done;
+	}
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..179886bc15b 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (!path_k)
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index 067cbe6990a..13807f58e95 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -38,3 +41,92 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR * dir;
+	int read;
+	int retval;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to;
+
+	retval = 0;
+	dir = opendir("/");
+	if (!dir)
+		return -1;
+
+	strbuf_init(&alias, 256);
+	strbuf_init(&points_to, MAXPATHLEN);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addch(&alias, '/');
+		strbuf_add(&alias, de->d_name, strlen(de->d_name));
+
+		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
+		if (read > 0) {
+			strbuf_setlen(&points_to, read);
+			if ((strncmp(points_to.buf, path, points_to.len) == 0)
+				&& path[points_to.len] == '/') {
+				info->alias = strbuf_detach(&alias, NULL);
+				info->points_to = strbuf_detach(&points_to, NULL);
+				trace_printf_key(&trace_fsmonitor,
+					"Found alias for '%s' : '%s' -> '%s'",
+					path, info->alias, info->points_to);
+				retval = 0;
+				goto done;
+			}
+		} else if (errno != EINVAL) { /* Something other than not a link */
+			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
+			retval = -1;
+			goto done;
+		}
+	}
+
+	done:
+	closedir(dir);
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	int len = info->alias ? strlen(info->alias) : 0;
+
+	if (!len)
+		return NULL;
+
+	if ((strncmp(info->alias, path, len) == 0)
+		&& path[len] == '/') {
+		struct strbuf tmp;
+		const char *remainder = path + len;
+		int ptr_len = strlen(info->points_to);
+		int rem_len = strlen(remainder);
+
+		strbuf_init(&tmp, ptr_len + rem_len);
+		strbuf_add(&tmp, info->points_to, ptr_len);
+		strbuf_add(&tmp, remainder, rem_len);
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index a90b8f7925b..0d95bbb416f 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..e24838f9a86 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index e48592887e7..50ef37e57bb 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
                                 ` (3 preceding siblings ...)
  2022-09-17  1:12               ` [PATCH v8 4/5] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-17  1:12               ` Eric DeCosta via GitGitGadget
  2022-09-17  6:08                 ` Eric Sunshine
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  5 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-17  1:12 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
Call-out experimental nature of 'fsmonitor.allowRemote' and limited file
system support for 'fsmonitor.socketDir'.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt
index cc142fb8612..0adccd0eced 100644
--- a/Documentation/git-fsmonitor--daemon.txt
+++ b/Documentation/git-fsmonitor--daemon.txt
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
@@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositores and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
+Usage of UDS requires the creation of a file which, by default, is created
+in the .git directory.  If the fsmonitor daemon detects that the .git directory
+is on a network-mounted file system, it will create the UDS file in $HOME.  If
+$HOME itself is on a network-mounted file system or if $HOME is not the desired
+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
+directory on a file system with proper support.  Mac OS native file systems have
+the required support.  File systems known to lack support include FAT32 and
+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
+daemon is not guaranteed to work with these file systems and such use is
+considered experimental.
+
+CONFIGURATION
+-------------
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+fsmonitor.allowRemote::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+fsmonitor.socketDir::
+	This option is only used by the Mac OS implementation of the fsmonitor
+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
+	directory on a file system that can support Unix domain sockets (UDS).
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  1:12               ` [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-17  4:13                 ` Eric Sunshine
  2022-09-19 16:50                   ` Junio C Hamano
  2022-09-17  6:29                 ` Eric Sunshine
  1 sibling, 1 reply; 157+ messages in thread
From: Eric Sunshine @ 2022-09-17  4:13 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> If the .git directory is on a remote file system, create the socket
> file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
> diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
> @@ -0,0 +1,9 @@
> +#include "config.h"
> +#include "fsmonitor-ipc.h"
> +
> +const char *fsmonitor_ipc__get_path(struct repository *r) {
> +       static char *ret;
> +       if (!ret)
> +               ret = git_pathdup("fsmonitor--daemon.ipc");
> +       return ret;
> +}
> \ No newline at end of file

This file probably wants a final line terminator.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-17  1:12               ` [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
@ 2022-09-17  6:08                 ` Eric Sunshine
  2022-09-19 23:55                   ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Eric Sunshine @ 2022-09-17  6:08 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
> Call-out experimental nature of 'fsmonitor.allowRemote' and limited file
> system support for 'fsmonitor.socketDir'.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
> diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt
> @@ -70,6 +70,41 @@ the change (as happening against the super repo).  However, the client
> +By default, the fsmonitor daemon refuses to work against network-mounted
> +repositories; this my be overridden by setting `fsmonitor.allowRemote` to
> +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
> +correctly with all network-mounted repositores and such use is considered
> +experimental.

s/repositores/repositories/

> +On Mac OS, the inter-process communication (IPC) between various Git
> +commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
> +Usage of UDS requires the creation of a file which, by default, is created
> +in the .git directory.  If the fsmonitor daemon detects that the .git directory

Typesetting: s/.git/`.git`/g

> +is on a network-mounted file system, it will create the UDS file in $HOME.  If

There's a gap in the explanation as to _why_ fsmonitor won't use the
.git directory for this file when on a network-mounted filesystem and
instead chooses $HOME. For the reader who is not well-versed in Unix
sockets/filesystems, it may be difficult to understand the logic
behind this. This gap is somewhat filled in a few sentences later, but
it makes for potentially confusing reading until then.

Should the reader know the name of the socket file or at least the
templated form of the name? The first question which popped into my
head upon reading this was whether it was going to pollute my home
directory with non-hidden files. If this had mentioned something along
the lines of "creation of a file named `.git-fsmonitor-*`" or
"creation of a hidden file" then I would have understood immediately
that the file would have been hidden.

> +$HOME itself is on a network-mounted file system or if $HOME is not the desired

To be consistent with formatting elsewhere in the Git documentation,
let's typeset this as `$HOME` (with backticks).

Aside: The spelling "filesystem" appears almost five times as often as
"file system" in Git documentation, however, this particular file
already uses "file system" and does so consistently, so it makes sense
to follow suit as you do here. Changing to use "filesystem" instead,
if such a task is desirable, is outside the scope of this patch
series.

> +location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local

For consistency, let's use backticks here, as well: `fsmonitor.socketDir`

> +directory on a file system with proper support.  Mac OS native file systems have

Together with the above comment about a gap in the explanation, I
found myself scratching my head about what "proper support" meant
(when pretending to read this as a person not particularly familiar
with Unix sockets or filesystems).

Also, although this explains how to work around the case when $HOME is
itself network-mounted, what happens if $HOME is network-mounted and
the user does not set `fsmonitor.socketDir`? Does it error out? Does
it simply misbehave in some way? Should it error out? (I would think
"yes".)

> +the required support.  File systems known to lack support include FAT32 and
> +NTFS.  Other file systems may or many not have the needed support; the fsmonitor

s/many/may/

> +daemon is not guaranteed to work with these file systems and such use is
> +considered experimental.

Taking the above comments into account, here's my attempt at a rewrite:

    On Mac OS, the inter-process communication (IPC) between various
    Git commands and the fsmonitor daemon occurs via a Unix domain
    socket (UDS) -- a special type of file -- which is supported by
    the native Mac OS filesystems but not by network-mounted
    filesystems, NTFS or FAT32.  Other file systems may or many not
    have the needed support; the fsmonitor daemon is not guaranteed to
    work with these file systems and such use is considered
    experimental.

    By default, the socket is created in the `.git` directory,
    however, if the `.git` directory is on a network-mounted file
    system, it will instead be created at `$HOME/.git-fsmonitor-*`
    unless `$HOME` itself is on a network-mounted file system, in
    which case you must set the configuration variable
    `fsmonitor.socketDir` to the path of a directory on a Mac OS
    native filesystem in which to create the socket file.

> +CONFIGURATION
> +-------------
> +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
> +the fsmonitor daemon will pay attention to the following configuration
> +variables:

We probably want a blank line after the header underline and before
this paragraph.

> +fsmonitor.allowRemote::
> +       By default, the daemon refuses to work against network-mounted
> +       repositories. Setting `fsmonitor.allowRemote` to `true` overrides
> +       this behavior.
> +
> +fsmonitor.socketDir::
> +       This option is only used by the Mac OS implementation of the fsmonitor
> +       daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local
> +       directory on a file system that can support Unix domain sockets (UDS).

Typeset with backticks: `fsmonitor.socketDir`

The word "valid" seems unnecessary. A possible rewrite:

    This Mac OS-specific option, if set, specifies the directory in
    which to create the Unix domain socket used for communication
    between fsmonitor and various Git commands. The directory must
    reside on a native Mac OS filesystem as discussed above.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  1:12               ` [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
  2022-09-17  4:13                 ` Eric Sunshine
@ 2022-09-17  6:29                 ` Eric Sunshine
  2022-09-17 16:29                   ` Eric DeCosta
  2022-09-19 16:58                   ` Junio C Hamano
  1 sibling, 2 replies; 157+ messages in thread
From: Eric Sunshine @ 2022-09-17  6:29 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> If the .git directory is on a remote file system, create the socket
> file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
> diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
> @@ -0,0 +1,46 @@
> +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
> +
> +const char *fsmonitor_ipc__get_path(struct repository *r)
> +{
> +       static const char *ipc_path;
> +       SHA_CTX sha1ctx;
> +       char *sock_dir;
> +       struct strbuf ipc_file = STRBUF_INIT;
> +       unsigned char hash[SHA_DIGEST_LENGTH];
> +
> +       if (ipc_path)
> +               return ipc_path;
> +
> +       ipc_path = fsmonitor_ipc__get_default_path();
> +
> +       /* By default the socket file is created in the .git directory */
> +       if (fsmonitor__is_fs_remote(ipc_path) < 1)
> +               return ipc_path;
> +
> +       SHA1_Init(&sha1ctx);
> +       SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
> +       SHA1_Final(hash, &sha1ctx);
> +
> +       repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
> +
> +       /* Create the socket file in either socketDir or $HOME */
> +       if (sock_dir && *sock_dir)
> +               strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> +                                       sock_dir, hash_to_hex(hash));
> +       else
> +               strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));

A couple comments...

In my mind, the directory specified by `fsmonitor.socketdir` is likely
to be dedicated to this purpose (i.e. housing Git administrative
junk). As such, it feels somewhat odd for the socket file to be
hidden; I would instead expect the socket name to be non-hidden (say,
"git-fsmonitor-daemon-{hash}.ipc") rather than hidden
(".git-fsmonitor-*"). The directory specified by `fsmonitor.socketdir`
may or may not be hidden (i.e. start with a dot), but that's the
user's decision. For the $HOME case, it almost feels cleaner to create
a hidden directory (say, "$HOME/.git-fsmonitor") in which to house the
socket files ("git-fsmonitor-daemon-{hash}.ipc"). Anyhow, this comment
is quite subjective; perhaps not actionable.

What happens if either $HOME or `fsmonitor.socketdir` are
network-mounted? Should this code be checking for that case? If they
are network-mounted, should it error out? At minimum, I would think a
warning is warranted in order to save users the headache of wondering
why fsmonitor isn't working correctly.

> +       ipc_path = interpolate_path(ipc_file.buf, 1);
> +       if (!ipc_path)
> +               die(_("Invalid path: %s"), ipc_file.buf);
> +
> +       strbuf_release(&ipc_file);

`sock_dir` is being leaked, isn't it?

> +       return ipc_path;
> +}
> diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
> @@ -0,0 +1,9 @@
> +const char *fsmonitor_ipc__get_path(struct repository *r) {
> +       static char *ret;
> +       if (!ret)
> +               ret = git_pathdup("fsmonitor--daemon.ipc");
> +       return ret;
> +}
> \ No newline at end of file

Mentioned already.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  6:29                 ` Eric Sunshine
@ 2022-09-17 16:29                   ` Eric DeCosta
  2022-09-19 16:58                   ` Junio C Hamano
  1 sibling, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-17 16:29 UTC (permalink / raw)
  To: Eric Sunshine, Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Eric Sunshine <sunshine@sunshineco.com>
> Sent: Saturday, September 17, 2022 2:30 AM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>
> Cc: Git List <git@vger.kernel.org>; Jeff Hostetler <git@jeffhostetler.com>;
> Torsten Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
> Johannes Schindelin <Johannes.Schindelin@gmx.de>; Eric DeCosta
> <edecosta@mathworks.com>
> Subject: Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is
> remote
> 
> On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > If the .git directory is on a remote file system, create the socket
> > file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
> >
> > Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> > ---
> > diff --git a/compat/fsmonitor/fsm-ipc-darwin.c
> > b/compat/fsmonitor/fsm-ipc-darwin.c
> > @@ -0,0 +1,46 @@
> > +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path,
> > +"fsmonitor--daemon.ipc")
> > +
> > +const char *fsmonitor_ipc__get_path(struct repository *r) {
> > +       static const char *ipc_path;
> > +       SHA_CTX sha1ctx;
> > +       char *sock_dir;
> > +       struct strbuf ipc_file = STRBUF_INIT;
> > +       unsigned char hash[SHA_DIGEST_LENGTH];
> > +
> > +       if (ipc_path)
> > +               return ipc_path;
> > +
> > +       ipc_path = fsmonitor_ipc__get_default_path();
> > +
> > +       /* By default the socket file is created in the .git directory */
> > +       if (fsmonitor__is_fs_remote(ipc_path) < 1)
> > +               return ipc_path;
> > +
> > +       SHA1_Init(&sha1ctx);
> > +       SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
> > +       SHA1_Final(hash, &sha1ctx);
> > +
> > +       repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
> > +
> > +       /* Create the socket file in either socketDir or $HOME */
> > +       if (sock_dir && *sock_dir)
> > +               strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> > +                                       sock_dir, hash_to_hex(hash));
> > +       else
> > +               strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s",
> > + hash_to_hex(hash));
> 
> A couple comments...
> 
> In my mind, the directory specified by `fsmonitor.socketdir` is likely to be
> dedicated to this purpose (i.e. housing Git administrative junk). As such, it
> feels somewhat odd for the socket file to be hidden; I would instead expect
> the socket name to be non-hidden (say,
> "git-fsmonitor-daemon-{hash}.ipc") rather than hidden (".git-fsmonitor-*").
> The directory specified by `fsmonitor.socketdir` may or may not be hidden
> (i.e. start with a dot), but that's the user's decision. For the $HOME case, it
> almost feels cleaner to create a hidden directory (say, "$HOME/.git-
> fsmonitor") in which to house the socket files ("git-fsmonitor-daemon-
> {hash}.ipc"). Anyhow, this comment is quite subjective; perhaps not
> actionable.

> What happens if either $HOME or `fsmonitor.socketdir` are network-
> mounted? Should this code be checking for that case? If they are network-
> mounted, should it error out? At minimum, I would think a warning is
> warranted in order to save users the headache of wondering why fsmonitor
> isn't working correctly.
> 
Ultimately, the UDS file location is checked by check_uds_volume in fsm-settings-darwin as part of the overall settings checks; it will error-out there if the path is on the network.

> > +       ipc_path = interpolate_path(ipc_file.buf, 1);
> > +       if (!ipc_path)
> > +               die(_("Invalid path: %s"), ipc_file.buf);
> > +
> > +       strbuf_release(&ipc_file);
> 
> `sock_dir` is being leaked, isn't it?
>

Sure is, thanks.


> > +       return ipc_path;
> > +}
> > diff --git a/compat/fsmonitor/fsm-ipc-win32.c
> > b/compat/fsmonitor/fsm-ipc-win32.c
> > @@ -0,0 +1,9 @@
> > +const char *fsmonitor_ipc__get_path(struct repository *r) {
> > +       static char *ret;
> > +       if (!ret)
> > +               ret = git_pathdup("fsmonitor--daemon.ipc");
> > +       return ret;
> > +}
> > \ No newline at end of file
> 
> Mentioned already.


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-16 20:11               ` Junio C Hamano
@ 2022-09-19 12:31                 ` Jeff Hostetler
  2022-09-19 16:42                   ` Junio C Hamano
  0 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-19 12:31 UTC (permalink / raw)
  To: Junio C Hamano, Eric DeCosta via GitGitGadget
  Cc: git, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta



On 9/16/22 4:11 PM, Junio C Hamano wrote:
> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> +const char *fsmonitor_ipc__get_path(struct repository *r)
>> +{
>> +	static const char *ipc_path;
>> +	SHA_CTX sha1ctx;
>> +	char *sock_dir;
>> +	struct strbuf ipc_file = STRBUF_INIT;
>> +	unsigned char hash[SHA_DIGEST_LENGTH];
>> +
>> +	if (ipc_path)
>> +		return ipc_path;
>> +
>> +	if (!r)
>> +		r = the_repository;
> 
> I'd prefer not to see this "NULL means the_repository".  It would be
> a different story if the caller does not necessarily have a ready
> access to the_repository, but it is a global, so the caller can pass
> the_repository and be more explicit.  Giving two ways to the caller
> to express same thing is not a good idea.
> 
> Thanks.
> 

To be fair, I added several "if (!r) r = the_repository;" statements
to the original public FSMonitor routines.  There were obscure cases
where tests would sometimes randomly fail because "r" wasn't completely
passed down via some hard to isolate call stack.  Offlist, AEvar told me
that he managed to isolate it and has a fix.

So eventually, we'll be able to get rid of all of these direct
references to "the_repository" and properly assume that "r" is
always passed down.

But for now, I think we should let this stay for safety.

Jeff


^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 12:31                 ` Jeff Hostetler
@ 2022-09-19 16:42                   ` Junio C Hamano
  2022-09-19 17:08                     ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-19 16:42 UTC (permalink / raw)
  To: Jeff Hostetler
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Jeff Hostetler <git@jeffhostetler.com> writes:

> On 9/16/22 4:11 PM, Junio C Hamano wrote:
>> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> 
>>> +const char *fsmonitor_ipc__get_path(struct repository *r)
>>> +{
>>> +	static const char *ipc_path;
>>> +	SHA_CTX sha1ctx;
>>> +	char *sock_dir;
>>> +	struct strbuf ipc_file = STRBUF_INIT;
>>> +	unsigned char hash[SHA_DIGEST_LENGTH];
>>> +
>>> +	if (ipc_path)
>>> +		return ipc_path;
>>> +
>>> +	if (!r)
>>> +		r = the_repository;
>> I'd prefer not to see this "NULL means the_repository".  It would be
>> a different story if the caller does not necessarily have a ready
>> access to the_repository, but it is a global, so the caller can pass
>> the_repository and be more explicit.  Giving two ways to the caller
>> to express same thing is not a good idea.
>> Thanks.
>> 
>
> To be fair, I added several "if (!r) r = the_repository;" statements
> to the original public FSMonitor routines.  There were obscure cases
> where tests would sometimes randomly fail because "r" wasn't completely
> passed down via some hard to isolate call stack.  Offlist, AEvar told me
> that he managed to isolate it and has a fix.
>
> So eventually, we'll be able to get rid of all of these direct
> references to "the_repository" and properly assume that "r" is
> always passed down.
>
> But for now, I think we should let this stay for safety.

I wouldn't call "sweeping a breakage under the rug" a "safety",
though.  If the caller cannot decide which repository instance is
the right thing to pass, or the caller does not yet have a good one
to pass when making a call down the codepath, how can it be safer to
use the_repository that may or may not be the appropriate one than
noticing the problem by dying and stopping the spread of damage?

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  4:13                 ` Eric Sunshine
@ 2022-09-19 16:50                   ` Junio C Hamano
  0 siblings, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-09-19 16:50 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Eric DeCosta via GitGitGadget, Git List, Jeff Hostetler,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> +}
>> \ No newline at end of file
>
> This file probably wants a final line terminator.

definitely ;-)

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
  2022-09-17  6:29                 ` Eric Sunshine
  2022-09-17 16:29                   ` Eric DeCosta
@ 2022-09-19 16:58                   ` Junio C Hamano
  1 sibling, 0 replies; 157+ messages in thread
From: Junio C Hamano @ 2022-09-19 16:58 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Eric DeCosta via GitGitGadget, Git List, Jeff Hostetler,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Eric Sunshine <sunshine@sunshineco.com> writes:

> A couple comments...
>
> In my mind, the directory specified by `fsmonitor.socketdir` is likely
> to be dedicated to this purpose (i.e. housing Git administrative
> junk). As such, it feels somewhat odd for the socket file to be
> hidden; I would instead expect the socket name to be non-hidden (say,
> "git-fsmonitor-daemon-{hash}.ipc") rather than hidden
> (".git-fsmonitor-*"). The directory specified by `fsmonitor.socketdir`
> may or may not be hidden (i.e. start with a dot), but that's the
> user's decision. For the $HOME case, it almost feels cleaner to create
> a hidden directory (say, "$HOME/.git-fsmonitor") in which to house the
> socket files ("git-fsmonitor-daemon-{hash}.ipc"). Anyhow, this comment
> is quite subjective; perhaps not actionable.

Yeah, dot-prefixed files are appropriate if they are to be placed at
the top of some tree without the user having any say in how that
tree is chosen (e.g. the working tree or $HOME).  If the user has
the power to specify the location, the equation changes.

> What happens if either $HOME or `fsmonitor.socketdir` are
> network-mounted? Should this code be checking for that case? If they
> are network-mounted, should it error out? At minimum, I would think a
> warning is warranted in order to save users the headache of wondering
> why fsmonitor isn't working correctly.

That's a good point.  If one default position (e.g. repository) is
checked if it is usable and can be rejected if it isn't, the
fallback position should at least satisfy the same "is it usable?"
criteria.

Thanks.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 16:42                   ` Junio C Hamano
@ 2022-09-19 17:08                     ` Jeff Hostetler
  2022-09-19 17:49                       ` Junio C Hamano
  0 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-19 17:08 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta



On 9/19/22 12:42 PM, Junio C Hamano wrote:
> Jeff Hostetler <git@jeffhostetler.com> writes:
> 
>> On 9/16/22 4:11 PM, Junio C Hamano wrote:
>>> "Eric DeCosta via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>>
>>>> +const char *fsmonitor_ipc__get_path(struct repository *r)
>>>> +{
>>>> +	static const char *ipc_path;
>>>> +	SHA_CTX sha1ctx;
>>>> +	char *sock_dir;
>>>> +	struct strbuf ipc_file = STRBUF_INIT;
>>>> +	unsigned char hash[SHA_DIGEST_LENGTH];
>>>> +
>>>> +	if (ipc_path)
>>>> +		return ipc_path;
>>>> +
>>>> +	if (!r)
>>>> +		r = the_repository;
>>> I'd prefer not to see this "NULL means the_repository".  It would be
>>> a different story if the caller does not necessarily have a ready
>>> access to the_repository, but it is a global, so the caller can pass
>>> the_repository and be more explicit.  Giving two ways to the caller
>>> to express same thing is not a good idea.
>>> Thanks.
>>>
>>
>> To be fair, I added several "if (!r) r = the_repository;" statements
>> to the original public FSMonitor routines.  There were obscure cases
>> where tests would sometimes randomly fail because "r" wasn't completely
>> passed down via some hard to isolate call stack.  Offlist, AEvar told me
>> that he managed to isolate it and has a fix.
>>
>> So eventually, we'll be able to get rid of all of these direct
>> references to "the_repository" and properly assume that "r" is
>> always passed down.
>>
>> But for now, I think we should let this stay for safety.
> 
> I wouldn't call "sweeping a breakage under the rug" a "safety",
> though.  If the caller cannot decide which repository instance is
> the right thing to pass, or the caller does not yet have a good one
> to pass when making a call down the codepath, how can it be safer to
> use the_repository that may or may not be the appropriate one than
> noticing the problem by dying and stopping the spread of damage?
> 

Aren't we in the middle of a transition from always
using the global "the_repository" to a passed "r" variable?
We're getting closer to being able to hide the the global
symbol, but we're not there yet, right?

I'm thinking that at as long as the global exists, we are not
safe to have multiple "struct repository" instances, right?

All I'm saying is that there are obscure/edge code paths where
a valid "r" is not being passed down.  Or, more likely, someone
has an "istate" that doesn't yet have "istate->repo" set at the
point of the call, so they might be passing "istate->repo", but
it is null.

Tracking down these nulls is important, but shouldn't it be
independent of this one?

Jeff

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 17:08                     ` Jeff Hostetler
@ 2022-09-19 17:49                       ` Junio C Hamano
  2022-09-19 23:51                         ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Junio C Hamano @ 2022-09-19 17:49 UTC (permalink / raw)
  To: Jeff Hostetler
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Jeff Hostetler <git@jeffhostetler.com> writes:

> Aren't we in the middle of a transition from always
> using the global "the_repository" to a passed "r" variable?
> We're getting closer to being able to hide the the global
> symbol, but we're not there yet, right?

We may still have code that works ONLY on the_repository, but
letting a function take "r" and lettin it ignore is worse than
leaving it explicitly limited to the_repository only, no?

> I'm thinking that at as long as the global exists, we are not
> safe to have multiple "struct repository" instances, right?

By itself, Not at all.  It is the code like I am criticizing that
makes it unsafe.

I do not mind adding

	if (!r)
		BUG(...);

at the place you have the "sweep it under the rug" band-aid, though.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-17  1:12             ` [PATCH v8 0/5] " Eric DeCosta via GitGitGadget
                                 ` (4 preceding siblings ...)
  2022-09-17  1:12               ` [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37               ` Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                                   ` (6 more replies)
  5 siblings, 7 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

v9 differs from v8:

 * incorporates code review feedback
 * check for incompatibility before communicating with fsmonitor

v8 differs from v7:

 * incorporates code review feedback
 * gets the rebase right

v7 differs from v6:

 * incorporates code review feedback

v6 differs from v5:

 * incorporates earlier, Windows-specific changes that have not made it back
   yet to the master branch
 * incorporates code review feedback
 * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'

v5 differs significantly from earlier versions:

 * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
   such that these options are no longer added to the settings data
   structure but are rather read from config at point of use
 * refactoring of code for handling platform-specific file system checks via
   a common interface to avoid platform #ifdef in IPC code and be in-model
   with other platform-specific fsmonitor code
 * dealing with 'synthetic firmlinks' on macOS

Eric DeCosta (6):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: check for compatability before communicating with fsmonitor
  fsmonitor: add documentation for allowRemote and socketDir options

 Documentation/git-fsmonitor--daemon.txt  |  48 ++++++-
 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |  11 +-
 compat/fsmonitor/fsm-ipc-darwin.c        |  48 +++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   9 ++
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 145 +++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  70 +++------
 compat/fsmonitor/fsm-settings-win32.c    | 174 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   3 +
 fsmonitor-ipc.c                          |  18 ++-
 fsmonitor-ipc.h                          |   4 +-
 fsmonitor-path-utils.h                   |  59 ++++++++
 fsmonitor-settings.c                     |  60 +++++++-
 fsmonitor-settings.h                     |   2 +-
 fsmonitor.c                              |   4 +
 18 files changed, 559 insertions(+), 240 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v9
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v9
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v8:

 1:  155a6890806 = 1:  155a6890806 fsmonitor: refactor filesystem checks to common interface
 2:  b5356497228 ! 2:  68709d788d5 fsmonitor: relocate socket file if .git directory is remote
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +	repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
      +
      +	/* Create the socket file in either socketDir or $HOME */
     -+	if (sock_dir && *sock_dir)
     ++	if (sock_dir && *sock_dir) {
      +		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
      +					sock_dir, hash_to_hex(hash));
     -+	else
     ++		free(sock_dir);
     ++	} else {
      +		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
     ++	}
      +
      +	ipc_path = interpolate_path(ipc_file.buf, 1);
      +	if (!ipc_path)
 3:  6719ca2b24d = 3:  6ddd922917a fsmonitor: avoid socket location check if using hook
 4:  d736cb8fa90 ! 4:  73afd9f3122 fsmonitor: deal with synthetic firmlinks on macOS
     @@ Commit message
      
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
     -    fsmonitor: deal with synthetic firmlinks on macOS
     -
     -    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
     -    called 'firmlinks' in order to separate the boot volume into two
     -    volumes, one read-only and one writable but still present them to the
     -    user as a single volume. Along with this change, Apple removed the
     -    ability to create symlinks in the root directory and replaced them with
     -    'synthetic firmlinks'. See 'man synthetic.conf'
     -
     -    When FSEevents reports the path of changed files, if the path involves
     -    a synthetic firmlink, the path is reported from the point of the
     -    synthetic firmlink and not the real path. For example:
     -
     -    Real path:
     -    /System/Volumes/Data/network/working/directory/foo.txt
     -
     -    Synthetic firmlink:
     -    /network -> /System/Volumes/Data/network
     -
     -    FSEvents path:
     -    /network/working/directory/foo.txt
     -
     -    This causes the FSEvents path to not match against the worktree
     -    directory.
     -
     -    There are several ways in which synthetic firmlinks can be created:
     -    they can be defined in /etc/synthetic.conf, the automounter can create
     -    them, and there may be other means. Simply reading /etc/synthetic.conf
     -    is insufficient. No matter what process creates synthetic firmlinks,
     -    they all get created in the root directory.
     -
     -    Therefore, in order to deal with synthetic firmlinks, the root directory
     -    is scanned and the first possible synthetic firmink that, when resolved,
     -    is a prefix of the worktree is used to map FSEvents paths to worktree
     -    paths.
     -
     -    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
     -
     -    fsmonitor: deal with synthetic firmlinks on macOS
     -
     -    Starting with macOS 10.15 (Catalina), Apple introduced a new feature
     -    called 'firmlinks' in order to separate the boot volume into two
     -    volumes, one read-only and one writable but still present them to the
     -    user as a single volume. Along with this change, Apple removed the
     -    ability to create symlinks in the root directory and replaced them with
     -    'synthetic firmlinks'. See 'man synthetic.conf'
     -
     -    When FSEevents reports the path of changed files, if the path involves
     -    a synthetic firmlink, the path is reported from the point of the
     -    synthetic firmlink and not the real path. For example:
     -
     -    Real path:
     -    /System/Volumes/Data/network/working/directory/foo.txt
     -
     -    Synthetic firmlink:
     -    /network -> /System/Volumes/Data/network
     -
     -    FSEvents path:
     -    /network/working/directory/foo.txt
     -
     -    This causes the FSEvents path to not match against the worktree
     -    directory.
     -
     -    There are several ways in which synthetic firmlinks can be created:
     -    they can be defined in /etc/synthetic.conf, the automounter can create
     -    them, and there may be other means. Simply reading /etc/synthetic.conf
     -    is insufficient. No matter what process creates synthetic firmlinks,
     -    they all get created in the root directory.
     -
     -    Therefore, in order to deal with synthetic firmlinks, the root directory
     -    is scanned and the first possible synthetic firmink that, when resolved,
     -    is a prefix of the worktree is used to map FSEvents paths to worktree
     -    paths.
     -
     -    Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
     -
       ## builtin/fsmonitor--daemon.c ##
      @@
       #include "parse-options.h"
 -:  ----------- > 5:  02afeaa01be fsmonitor: check for compatability before communicating with fsmonitor
 5:  ddf4e3e6442 ! 6:  0e8ea28acc1 fsmonitor: add documentation for allowRemote and socketDir options
     @@ Commit message
          Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
      
       ## Documentation/git-fsmonitor--daemon.txt ##
     -@@ Documentation/git-fsmonitor--daemon.txt: the change (as happening against the super repo).  However, the client
     +@@ Documentation/git-fsmonitor--daemon.txt: git-fsmonitor{litdd}daemon(1)
     + 
     + NAME
     + ----
     +-git-fsmonitor--daemon - A Built-in File System Monitor
     ++git-fsmonitor--daemon - A Built-in Filesystem Monitor
     + 
     + SYNOPSIS
     + --------
     +@@ Documentation/git-fsmonitor--daemon.txt: DESCRIPTION
     + -----------
     + 
     + A daemon to watch the working directory for file and directory
     +-changes using platform-specific file system notification facilities.
     ++changes using platform-specific filesystem notification facilities.
     + 
     + This daemon communicates directly with commands like `git status`
     + using the link:technical/api-simple-ipc.html[simple IPC] interface
     +@@ Documentation/git-fsmonitor--daemon.txt: CAVEATS
     + -------
     + 
     + The fsmonitor daemon does not currently know about submodules and does
     +-not know to filter out file system events that happen within a
     ++not know to filter out filesystem events that happen within a
     + submodule.  If fsmonitor daemon is watching a super repo and a file is
     + modified within the working directory of a submodule, it will report
     + the change (as happening against the super repo).  However, the client
       will properly ignore these extra events, so performance may be affected
       but it will not cause an incorrect result.
       
      +By default, the fsmonitor daemon refuses to work against network-mounted
     -+repositories; this my be overridden by setting `fsmonitor.allowRemote` to
     ++repositories; this may be overridden by setting `fsmonitor.allowRemote` to
      +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
     -+correctly with all network-mounted repositores and such use is considered
     ++correctly with all network-mounted repositories and such use is considered
      +experimental.
      +
      +On Mac OS, the inter-process communication (IPC) between various Git
     -+commands and the fsmonitor daemon is done via a Unix domain socket (UDS).
     -+Usage of UDS requires the creation of a file which, by default, is created
     -+in the .git directory.  If the fsmonitor daemon detects that the .git directory
     -+is on a network-mounted file system, it will create the UDS file in $HOME.  If
     -+$HOME itself is on a network-mounted file system or if $HOME is not the desired
     -+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
     -+directory on a file system with proper support.  Mac OS native file systems have
     -+the required support.  File systems known to lack support include FAT32 and
     -+NTFS.  Other file systems may or many not have the needed support; the fsmonitor
     -+daemon is not guaranteed to work with these file systems and such use is
     -+considered experimental.
     ++commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a
     ++special type of file -- which is supported by native Mac OS filesystems,
     ++but not on network-mounted filesystems, NTFS, or FAT32.  Other filesystems
     ++may or may not have the needed support; the fsmonitor daemon is not guaranteed
     ++to work with these filesystems and such use is considered experimental.
     ++
     ++By default, the socket is created in the `.git` directory, however, if the
     ++`.git` directory is on a network-mounted filesystem, it will be instead be
     ++created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a
     ++network-mounted filesystem in which case you must set the configuration
     ++variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native
     ++filesystem in which to create the socket file.
     ++
     ++If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`)
     ++is on a native Mac OS file filesystem the fsmonitor daemon will report an
     ++error that will cause the daemon and the currently running command to exit.
      +
      +CONFIGURATION
      +-------------
     ++
      +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
      +the fsmonitor daemon will pay attention to the following configuration
      +variables:
      +
     -+fsmonitor.allowRemote::
     ++`fsmonitor.allowRemote`::
      +	By default, the daemon refuses to work against network-mounted
      +	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
      +	this behavior.
      +
     -+fsmonitor.socketDir::
     -+	This option is only used by the Mac OS implementation of the fsmonitor
     -+	daemon.	If set, 'fsmonitor.socketDir' must be set to a valid, local
     -+	directory on a file system that can support Unix domain sockets (UDS).
     ++`fsmonitor.socketDir`::
     ++    This Mac OS-specific option, if set, specifies the directory in
     ++    which to create the Unix domain socket used for communication
     ++    between fsmonitor and various Git commands. The directory must
     ++    reside on a native Mac OS filesystem as discussed above.
      +
       GIT
       ---

-- 
gitgitgadget

^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v9 1/6] fsmonitor: refactor filesystem checks to common interface
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
                                   ` (5 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Provide a common interface for getting basic filesystem information
including filesystem type and whether the filesystem is remote.

Refactor existing code for getting basic filesystem info and detecting
remote file systems to the new interface.

Refactor filesystem checks to leverage new interface. For macOS,
error-out if the Unix Domain socket (UDS) file is on a remote
filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                                 |   1 +
 compat/fsmonitor/fsm-path-utils-darwin.c |  40 ++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 128 +++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  62 +++-----
 compat/fsmonitor/fsm-settings-win32.c    | 172 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   2 +
 fsmonitor-path-utils.h                   |  23 +++
 fsmonitor-settings.c                     |  50 +++++++
 8 files changed, 263 insertions(+), 215 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h

diff --git a/Makefile b/Makefile
index d9247ead45b..6e492a67547 100644
--- a/Makefile
+++ b/Makefile
@@ -2039,6 +2039,7 @@ endif
 ifdef FSMONITOR_OS_SETTINGS
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS
 	COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o
 endif
 
 ifeq ($(TCLTK_PATH),)
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
new file mode 100644
index 00000000000..067cbe6990a
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -0,0 +1,40 @@
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+#include <sys/param.h>
+#include <sys/mount.h>
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	struct statfs fs;
+	if (statfs(path, &fs) == -1) {
+		int saved_errno = errno;
+		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+				 path, strerror(saved_errno));
+		errno = saved_errno;
+		return -1;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
+			 path, fs.f_type, fs.f_flags, fs.f_fstypename);
+
+	if (!(fs.f_flags & MNT_LOCAL))
+		fs_info->is_remote = 1;
+	else
+		fs_info->is_remote = 0;
+
+	fs_info->typename = fs.f_fstypename;
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
new file mode 100644
index 00000000000..a90b8f7925b
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -0,0 +1,128 @@
+#include "cache.h"
+#include "fsmonitor.h"
+#include "fsmonitor-path-utils.h"
+
+/*
+ * Check remote working directory protocol.
+ *
+ * Return -1 if client machine cannot get remote protocol information.
+ */
+static int check_remote_protocol(wchar_t *wpath)
+{
+	HANDLE h;
+	FILE_REMOTE_PROTOCOL_INFO proto_info;
+
+	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+			FILE_FLAG_BACKUP_SEMANTICS, NULL);
+
+	if (h == INVALID_HANDLE_VALUE) {
+		error(_("[GLE %ld] unable to open for read '%ls'"),
+		      GetLastError(), wpath);
+		return -1;
+	}
+
+	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+		&proto_info, sizeof(proto_info))) {
+		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+		      GetLastError(), wpath);
+		CloseHandle(h);
+		return -1;
+	}
+
+	CloseHandle(h);
+
+	trace_printf_key(&trace_fsmonitor,
+				"check_remote_protocol('%ls') remote protocol %#8.8lx",
+				wpath, proto_info.Protocol);
+
+	return 0;
+}
+
+/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
+ *     (This is the normal method to access it.)
+ *
+ *     $ NET USE Z: \\server\share
+ *     $ git -C Z:/repo status
+ *
+ * (b) Windows allows a network share to be referenced WITHOUT mapping
+ *     it to drive letter.
+ *
+ *     $ NET USE \\server\share\dir
+ *     $ git -C //server/share/repo status
+ *
+ * (c) Windows allows "SUBST" to create a fake drive mapping to an
+ *     arbitrary path (which may be remote)
+ *
+ *     $ SUBST Q: Z:\repo
+ *     $ git -C Q:/ status
+ *
+ * (d) Windows allows a directory symlink to be created on a local
+ *     file system that points to a remote repo.
+ *
+ *     $ mklink /d ./link //server/share/repo
+ *     $ git -C ./link status
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+	wchar_t wpath[MAX_PATH];
+	wchar_t wfullpath[MAX_PATH];
+	size_t wlen;
+	UINT driveType;
+
+	/*
+	 * Do everything in wide chars because the drive letter might be
+	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
+	 */
+	if (xutftowcs_path(wpath, path) < 0) {
+		return -1;
+	}
+
+	/*
+	 * GetDriveTypeW() requires a final slash.  We assume that the
+	 * worktree pathname points to an actual directory.
+	 */
+	wlen = wcslen(wpath);
+	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
+		wpath[wlen++] = L'\\';
+		wpath[wlen] = 0;
+	}
+
+	/*
+	 * Normalize the path.  If nothing else, this converts forward
+	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
+	 * correctly handle some UNC "\\server\share\..." paths.
+	 */
+	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL)) {
+		return -1;
+	}
+
+	driveType = GetDriveTypeW(wfullpath);
+	trace_printf_key(&trace_fsmonitor,
+			 "DriveType '%s' L'%ls' (%u)",
+			 path, wfullpath, driveType);
+
+	if (driveType == DRIVE_REMOTE) {
+		fs_info->is_remote = 1;
+		if (check_remote_protocol(wfullpath) < 0)
+			return -1;
+	} else {
+		fs_info->is_remote = 0;
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+				"'%s' is_remote: %d",
+				path, fs_info->is_remote);
+
+	return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+	struct fs_info fs;
+	if (fsmonitor__get_fs_info(path, &fs))
+		return -1;
+	return fs.is_remote;
+}
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index efc732c0f31..dba3ced6bb7 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -1,32 +1,10 @@
-#include "cache.h"
 #include "config.h"
-#include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
-#include <sys/param.h>
-#include <sys/mount.h>
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
-/*
- * [1] Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type (NFS, SAMBA, etc.) dictates whether notification events
- * are available at all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
+ /*
  * For the builtin FSMonitor, we create the Unix domain socket for the
  * IPC in the .git directory.  If the working directory is remote,
  * then the socket will be created on the remote file system.  This
@@ -38,40 +16,34 @@
  * be taken to ensure that $HOME is actually local and not a managed
  * file share.)
  *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
+ * FAT32 and NTFS working directories are problematic too.
  *
  * The builtin FSMonitor uses a Unix domain socket in the .git
  * directory for IPC.  These Windows drive formats do not support
  * Unix domain sockets, so mark them as incompatible for the daemon.
  *
  */
-static enum fsmonitor_reason check_volume(struct repository *r)
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
-	struct statfs fs;
+	struct fs_info fs;
+	const char *ipc_path = fsmonitor_ipc__get_path();
+	struct strbuf path = STRBUF_INIT;
+	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
-	if (statfs(r->worktree, &fs) == -1) {
-		int saved_errno = errno;
-		trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-				 r->worktree, strerror(saved_errno));
-		errno = saved_errno;
+	if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) {
+		strbuf_release(&path);
 		return FSMONITOR_REASON_ERROR;
 	}
 
-	trace_printf_key(&trace_fsmonitor,
-			 "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-			 r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
+	strbuf_release(&path);
 
-	if (!(fs.f_flags & MNT_LOCAL))
+	if (fs.is_remote)
 		return FSMONITOR_REASON_REMOTE;
 
-	if (!strcmp(fs.f_fstypename, "msdos")) /* aka FAT32 */
+	if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
 		return FSMONITOR_REASON_NOSOCKETS;
 
-	if (!strcmp(fs.f_fstypename, "ntfs"))
+	if (!strcmp(fs.typename, "ntfs"))
 		return FSMONITOR_REASON_NOSOCKETS;
 
 	return FSMONITOR_REASON_OK;
@@ -81,7 +53,7 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_volume(r);
+	reason = check_uds_volume(r);
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index e5ec5b0a9f7..d88b06ae610 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -1,8 +1,9 @@
 #include "cache.h"
 #include "config.h"
 #include "repository.h"
-#include "fsmonitor-settings.h"
 #include "fsmonitor.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * VFS for Git is incompatible with FSMonitor.
@@ -24,171 +25,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-/*
- * Check if monitoring remote working directories is allowed.
- *
- * By default, monitoring remote working directories is
- * disabled.  Users may override this behavior in enviroments where
- * they have proper support.
- */
-static int check_config_allowremote(struct repository *r)
-{
-	int allow;
-
-	if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-		return allow;
-
-	return -1; /* fsmonitor.allowremote not set */
-}
-
-/*
- * Check remote working directory protocol.
- *
- * Error if client machine cannot get remote protocol information.
- */
-static int check_remote_protocol(wchar_t *wpath)
-{
-	HANDLE h;
-	FILE_REMOTE_PROTOCOL_INFO proto_info;
-
-	h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
-			FILE_FLAG_BACKUP_SEMANTICS, NULL);
-
-	if (h == INVALID_HANDLE_VALUE) {
-		error(_("[GLE %ld] unable to open for read '%ls'"),
-		      GetLastError(), wpath);
-		return -1;
-	}
-
-	if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
-		&proto_info, sizeof(proto_info))) {
-		error(_("[GLE %ld] unable to get protocol information for '%ls'"),
-		      GetLastError(), wpath);
-		CloseHandle(h);
-		return -1;
-	}
-
-	CloseHandle(h);
-
-	trace_printf_key(&trace_fsmonitor,
-				"check_remote_protocol('%ls') remote protocol %#8.8lx",
-				wpath, proto_info.Protocol);
-
-	return 0;
-}
-
-/*
- * Remote working directories are problematic for FSMonitor.
- *
- * The underlying file system on the server machine and/or the remote
- * mount type dictates whether notification events are available at
- * all to remote client machines.
- *
- * Kernel differences between the server and client machines also
- * dictate the how (buffering, frequency, de-dup) the events are
- * delivered to client machine processes.
- *
- * A client machine (such as a laptop) may choose to suspend/resume
- * and it is unclear (without lots of testing) whether the watcher can
- * resync after a resume.  We might be able to treat this as a normal
- * "events were dropped by the kernel" event and do our normal "flush
- * and resync" --or-- we might need to close the existing (zombie?)
- * notification fd and create a new one.
- *
- * In theory, the above issues need to be addressed whether we are
- * using the Hook or IPC API.
- *
- * So (for now at least), mark remote working directories as
- * incompatible.
- *
- * Notes for testing:
- *
- * (a) Windows allows a network share to be mapped to a drive letter.
- *     (This is the normal method to access it.)
- *
- *     $ NET USE Z: \\server\share
- *     $ git -C Z:/repo status
- *
- * (b) Windows allows a network share to be referenced WITHOUT mapping
- *     it to drive letter.
- *
- *     $ NET USE \\server\share\dir
- *     $ git -C //server/share/repo status
- *
- * (c) Windows allows "SUBST" to create a fake drive mapping to an
- *     arbitrary path (which may be remote)
- *
- *     $ SUBST Q: Z:\repo
- *     $ git -C Q:/ status
- *
- * (d) Windows allows a directory symlink to be created on a local
- *     file system that points to a remote repo.
- *
- *     $ mklink /d ./link //server/share/repo
- *     $ git -C ./link status
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
-	int ret;
-	wchar_t wpath[MAX_PATH];
-	wchar_t wfullpath[MAX_PATH];
-	size_t wlen;
-	UINT driveType;
-
-	/*
-	 * Do everything in wide chars because the drive letter might be
-	 * a multi-byte sequence.  See win32_has_dos_drive_prefix().
-	 */
-	if (xutftowcs_path(wpath, r->worktree) < 0)
-		return FSMONITOR_REASON_ERROR;
-
-	/*
-	 * GetDriveTypeW() requires a final slash.  We assume that the
-	 * worktree pathname points to an actual directory.
-	 */
-	wlen = wcslen(wpath);
-	if (wpath[wlen - 1] != L'\\' && wpath[wlen - 1] != L'/') {
-		wpath[wlen++] = L'\\';
-		wpath[wlen] = 0;
-	}
-
-	/*
-	 * Normalize the path.  If nothing else, this converts forward
-	 * slashes to backslashes.  This is essential to get GetDriveTypeW()
-	 * correctly handle some UNC "\\server\share\..." paths.
-	 */
-	if (!GetFullPathNameW(wpath, MAX_PATH, wfullpath, NULL))
-		return FSMONITOR_REASON_ERROR;
-
-	driveType = GetDriveTypeW(wfullpath);
-	trace_printf_key(&trace_fsmonitor,
-			 "DriveType '%s' L'%ls' (%u)",
-			 r->worktree, wfullpath, driveType);
-
-	if (driveType == DRIVE_REMOTE) {
-		trace_printf_key(&trace_fsmonitor,
-				 "check_remote('%s') true",
-				 r->worktree);
-
-		ret = check_remote_protocol(wfullpath);
-		if (ret < 0)
-			return FSMONITOR_REASON_ERROR;
-
-		switch (check_config_allowremote(r)) {
-		case 0: /* config overrides and disables */
-			return FSMONITOR_REASON_REMOTE;
-		case 1: /* config overrides and enables */
-			return FSMONITOR_REASON_OK;
-		default:
-			break; /* config has no opinion */
-		}
-
-		return FSMONITOR_REASON_REMOTE;
-	}
-
-	return FSMONITOR_REASON_OK;
-}
-
 enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 {
 	enum fsmonitor_reason reason;
@@ -197,9 +33,5 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
 	if (reason != FSMONITOR_REASON_OK)
 		return reason;
 
-	reason = check_remote(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
-
 	return FSMONITOR_REASON_OK;
 }
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 2237109b57f..b88494bf59b 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
new file mode 100644
index 00000000000..e48592887e7
--- /dev/null
+++ b/fsmonitor-path-utils.h
@@ -0,0 +1,23 @@
+#ifndef FSM_PATH_UTILS_H
+#define FSM_PATH_UTILS_H
+
+struct fs_info {
+	int is_remote;
+	char *typename;
+};
+
+/*
+ * Get some basic filesystem informtion for the given path
+ *
+ * Returns -1 on error, zero otherwise.
+ */
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+
+/*
+ * Determines if the filesystem that path resides on is remote.
+ *
+ * Returns -1 on error, 0 if not remote, 1 if remote.
+ */
+int fsmonitor__is_fs_remote(const char *path);
+
+#endif
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 464424a1e92..d288cbad479 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
@@ -2,6 +2,7 @@
 #include "config.h"
 #include "repository.h"
 #include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
 
 /*
  * We keep this structure defintion private and have getters
@@ -13,6 +14,52 @@ struct fsmonitor_settings {
 	char *hook_path;
 };
 
+/*
+ * Remote working directories are problematic for FSMonitor.
+ *
+ * The underlying file system on the server machine and/or the remote
+ * mount type dictates whether notification events are available at
+ * all to remote client machines.
+ *
+ * Kernel differences between the server and client machines also
+ * dictate the how (buffering, frequency, de-dup) the events are
+ * delivered to client machine processes.
+ *
+ * A client machine (such as a laptop) may choose to suspend/resume
+ * and it is unclear (without lots of testing) whether the watcher can
+ * resync after a resume.  We might be able to treat this as a normal
+ * "events were dropped by the kernel" event and do our normal "flush
+ * and resync" --or-- we might need to close the existing (zombie?)
+ * notification fd and create a new one.
+ *
+ * In theory, the above issues need to be addressed whether we are
+ * using the Hook or IPC API.
+ *
+ * So (for now at least), mark remote working directories as
+ * incompatible unless 'fsmonitor.allowRemote' is true.
+ *
+ */
+#ifdef HAVE_FSMONITOR_OS_SETTINGS
+static enum fsmonitor_reason check_remote(struct repository *r)
+{
+	int allow_remote = -1; /* -1 unset, 0 not allowed, 1 allowed */
+	int is_remote = fsmonitor__is_fs_remote(r->worktree);
+
+	switch (is_remote) {
+		case 0:
+			return FSMONITOR_REASON_OK;
+		case 1:
+			repo_config_get_bool(r, "fsmonitor.allowremote", &allow_remote);
+			if (allow_remote < 1)
+				return FSMONITOR_REASON_REMOTE;
+			else
+				return FSMONITOR_REASON_OK;
+		default:
+			return FSMONITOR_REASON_ERROR;
+	}
+}
+#endif
+
 static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 {
 	if (!r->worktree) {
@@ -27,6 +74,9 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 	{
 		enum fsmonitor_reason reason;
 
+		reason = check_remote(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
 		reason = fsm_os__incompatible(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-19 19:57                   ` Eric Sunshine
  2022-09-19 19:37                 ` [PATCH v9 3/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
                                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If the .git directory is on a remote file system, create the socket
file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Makefile                               |  1 +
 builtin/fsmonitor--daemon.c            |  3 +-
 compat/fsmonitor/fsm-ipc-darwin.c      | 48 ++++++++++++++++++++++++++
 compat/fsmonitor/fsm-ipc-win32.c       |  9 +++++
 compat/fsmonitor/fsm-settings-darwin.c |  2 +-
 contrib/buildsystems/CMakeLists.txt    |  2 ++
 fsmonitor-ipc.c                        | 18 +++++-----
 fsmonitor-ipc.h                        |  4 ++-
 8 files changed, 74 insertions(+), 13 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c

diff --git a/Makefile b/Makefile
index 6e492a67547..58bb9248471 100644
--- a/Makefile
+++ b/Makefile
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND
 	COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
 	COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
 	COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
+	COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
 endif
 
 ifdef FSMONITOR_OS_SETTINGS
diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 2c109cf8b37..0123fc33ed2 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)
 	 * directory.)
 	 */
 	strbuf_init(&state.path_ipc, 0);
-	strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path()));
+	strbuf_addstr(&state.path_ipc,
+		absolute_path(fsmonitor_ipc__get_path(the_repository)));
 
 	/*
 	 * Confirm that we can create platform-specific resources for the
diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
new file mode 100644
index 00000000000..d48d67690ee
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-darwin.c
@@ -0,0 +1,48 @@
+#include "cache.h"
+#include "config.h"
+#include "strbuf.h"
+#include "fsmonitor.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(struct repository *r)
+{
+	static const char *ipc_path;
+	SHA_CTX sha1ctx;
+	char *sock_dir;
+	struct strbuf ipc_file = STRBUF_INIT;
+	unsigned char hash[SHA_DIGEST_LENGTH];
+
+	if (ipc_path)
+		return ipc_path;
+
+	ipc_path = fsmonitor_ipc__get_default_path();
+
+	/* By default the socket file is created in the .git directory */
+	if (fsmonitor__is_fs_remote(ipc_path) < 1)
+		return ipc_path;
+
+	SHA1_Init(&sha1ctx);
+	SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
+	SHA1_Final(hash, &sha1ctx);
+
+	repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+	/* Create the socket file in either socketDir or $HOME */
+	if (sock_dir && *sock_dir) {
+		strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+					sock_dir, hash_to_hex(hash));
+		free(sock_dir);
+	} else {
+		strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+	}
+
+	ipc_path = interpolate_path(ipc_file.buf, 1);
+	if (!ipc_path)
+		die(_("Invalid path: %s"), ipc_file.buf);
+
+	strbuf_release(&ipc_file);
+	return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
new file mode 100644
index 00000000000..3a3a46db209
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-win32.c
@@ -0,0 +1,9 @@
+#include "config.h"
+#include "fsmonitor-ipc.h"
+
+const char *fsmonitor_ipc__get_path(struct repository *r) {
+	static char *ret;
+	if (!ret)
+		ret = git_pathdup("fsmonitor--daemon.ipc");
+	return ret;
+}
\ No newline at end of file
diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index dba3ced6bb7..681d8bf963e 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -26,7 +26,7 @@
 static enum fsmonitor_reason check_uds_volume(struct repository *r)
 {
 	struct fs_info fs;
-	const char *ipc_path = fsmonitor_ipc__get_path();
+	const char *ipc_path = fsmonitor_ipc__get_path(r);
 	struct strbuf path = STRBUF_INIT;
 	strbuf_add(&path, ipc_path, strlen(ipc_path));
 
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index b88494bf59b..7e7b6b9a362 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC)
 		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c
index 789e7397baa..c0f42301c84 100644
--- a/fsmonitor-ipc.c
+++ b/fsmonitor-ipc.c
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)
 	return 0;
 }
 
-const char *fsmonitor_ipc__get_path(void)
+const char *fsmonitor_ipc__get_path(struct repository *r)
 {
 	return NULL;
 }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)
 	return 1;
 }
 
-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-
 enum ipc_active_state fsmonitor_ipc__get_state(void)
 {
-	return ipc_get_active_state(fsmonitor_ipc__get_path());
+	return ipc_get_active_state(fsmonitor_ipc__get_path(the_repository));
 }
 
 static int spawn_daemon(void)
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,
 	trace2_data_string("fsm_client", NULL, "query/command", tok);
 
 try_again:
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 
 	switch (state) {
 	case IPC_STATE__LISTENING:
@@ -117,13 +115,13 @@ try_again:
 
 	case IPC_STATE__INVALID_PATH:
 		ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 
 	case IPC_STATE__OTHER_ERROR:
 	default:
 		ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
-			    fsmonitor_ipc__get_path());
+			    fsmonitor_ipc__get_path(the_repository));
 		goto done;
 	}
 
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command,
 	options.wait_if_busy = 1;
 	options.wait_if_not_found = 0;
 
-	state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
-				       &connection);
+	state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
+						&options, &connection);
 	if (state != IPC_STATE__LISTENING) {
 		die(_("fsmonitor--daemon is not running"));
 		return -1;
diff --git a/fsmonitor-ipc.h b/fsmonitor-ipc.h
index b6a7067c3af..8b489da762b 100644
--- a/fsmonitor-ipc.h
+++ b/fsmonitor-ipc.h
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
@@ -3,6 +3,8 @@
 
 #include "simple-ipc.h"
 
+struct repository;
+
 /*
  * Returns true if built-in file system monitor daemon is defined
  * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);
  *
  * Returns NULL if the daemon is not supported on this platform.
  */
-const char *fsmonitor_ipc__get_path(void);
+const char *fsmonitor_ipc__get_path(struct repository *r);
 
 /*
  * Try to determine whether there is a `git-fsmonitor--daemon` process
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v9 3/6] fsmonitor: avoid socket location check if using hook
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 4/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
                                   ` (3 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If monitoring is done via fsmonitor hook rather than IPC there is no
need to check if the location of the Unix Domain socket (UDS) file is
on a remote filesystem.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++----
 compat/fsmonitor/fsm-settings-win32.c  |  2 +-
 fsmonitor-settings.c                   |  8 ++++----
 fsmonitor-settings.h                   |  2 +-
 4 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-darwin.c
index 681d8bf963e..40da2d3b533 100644
--- a/compat/fsmonitor/fsm-settings-darwin.c
+++ b/compat/fsmonitor/fsm-settings-darwin.c
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
-	reason = check_uds_volume(r);
-	if (reason != FSMONITOR_REASON_OK)
-		return reason;
+	if (ipc) {
+		reason = check_uds_volume(r);
+		if (reason != FSMONITOR_REASON_OK)
+			return reason;
+	}
 
 	return FSMONITOR_REASON_OK;
 }
diff --git a/compat/fsmonitor/fsm-settings-win32.c b/compat/fsmonitor/fsm-settings-win32.c
index d88b06ae610..a8af31b71de 100644
--- a/compat/fsmonitor/fsm-settings-win32.c
+++ b/compat/fsmonitor/fsm-settings-win32.c
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)
 	return FSMONITOR_REASON_OK;
 }
 
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
 {
 	enum fsmonitor_reason reason;
 
diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index d288cbad479..531a1b6f956 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r)
 }
 #endif
 
-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
 {
 	if (!r->worktree) {
 		/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r)
 		reason = check_remote(r);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
-		reason = fsm_os__incompatible(r);
+		reason = fsm_os__incompatible(r, ipc);
 		if (reason != FSMONITOR_REASON_OK)
 			return reason;
 	}
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)
 
 void fsm_settings__set_ipc(struct repository *r)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 1);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)
 
 void fsm_settings__set_hook(struct repository *r, const char *path)
 {
-	enum fsmonitor_reason reason = check_for_incompatible(r);
+	enum fsmonitor_reason reason = check_for_incompatible(r, 0);
 
 	if (reason != FSMONITOR_REASON_OK) {
 		fsm_settings__set_incompatible(r, reason);
diff --git a/fsmonitor-settings.h b/fsmonitor-settings.h
index d9c2605197f..0721617b95a 100644
--- a/fsmonitor-settings.h
+++ b/fsmonitor-settings.h
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
@@ -48,7 +48,7 @@ struct fsmonitor_settings;
  * fsm_os__* routines should considered private to fsm_settings__
  * routines.
  */
-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
 #endif /* HAVE_FSMONITOR_OS_SETTINGS */
 
 #endif /* FSMONITOR_SETTINGS_H */
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v9 4/6] fsmonitor: deal with synthetic firmlinks on macOS
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                                   ` (2 preceding siblings ...)
  2022-09-19 19:37                 ` [PATCH v9 3/6] fsmonitor: avoid socket location check if using hook Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 5/6] fsmonitor: check for compatability before communicating with fsmonitor Eric DeCosta via GitGitGadget
                                   ` (2 subsequent siblings)
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Starting with macOS 10.15 (Catalina), Apple introduced a new feature
called 'firmlinks' in order to separate the boot volume into two
volumes, one read-only and one writable but still present them to the
user as a single volume. Along with this change, Apple removed the
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'

When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:

Real path:
/System/Volumes/Data/network/working/directory/foo.txt

Synthetic firmlink:
/network -> /System/Volumes/Data/network

FSEvents path:
/network/working/directory/foo.txt

This causes the FSEvents path to not match against the worktree
directory.

There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create
them, and there may be other means. Simply reading /etc/synthetic.conf
is insufficient. No matter what process creates synthetic firmlinks,
they all get created in the root directory.

Therefore, in order to deal with synthetic firmlinks, the root directory
is scanned and the first possible synthetic firmink that, when resolved,
is a prefix of the worktree is used to map FSEvents paths to worktree
paths.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 builtin/fsmonitor--daemon.c              |  8 +++
 compat/fsmonitor/fsm-listen-darwin.c     |  6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 17 +++++
 fsmonitor--daemon.h                      |  3 +
 fsmonitor-path-utils.h                   | 36 ++++++++++
 6 files changed, 161 insertions(+), 1 deletion(-)

diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c
index 0123fc33ed2..56fcd1c2baa 100644
--- a/builtin/fsmonitor--daemon.c
+++ b/builtin/fsmonitor--daemon.c
@@ -3,6 +3,7 @@
 #include "parse-options.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
 #include "compat/fsmonitor/fsm-health.h"
 #include "compat/fsmonitor/fsm-listen.h"
 #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)
 	strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
 	state.nr_paths_watching = 1;
 
+	state.alias.alias = NULL;
+	state.alias.points_to = NULL;
+	if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+		err = error(_("could not get worktree alias"));
+		goto done;
+	}
+
 	/*
 	 * We create and delete cookie files somewhere inside the .git
 	 * directory to help us keep sync with the file system.  If
diff --git a/compat/fsmonitor/fsm-listen-darwin.c b/compat/fsmonitor/fsm-listen-darwin.c
index 8e208e8289e..179886bc15b 100644
--- a/compat/fsmonitor/fsm-listen-darwin.c
+++ b/compat/fsmonitor/fsm-listen-darwin.c
@@ -26,6 +26,7 @@
 #include "fsmonitor.h"
 #include "fsm-listen.h"
 #include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsm_listen_data
 {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 		/*
 		 * On Mac, we receive an array of absolute paths.
 		 */
-		path_k = paths[k];
+		path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
+		if (!path_k)
+			path_k = paths[k];
 
 		/*
 		 * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,
 			fsmonitor_force_resync(state);
 			fsmonitor_batch__free_list(batch);
 			string_list_clear(&cookie_list, 0);
+			batch = NULL;
 
 			/*
 			 * We assume that any events that we received
diff --git a/compat/fsmonitor/fsm-path-utils-darwin.c b/compat/fsmonitor/fsm-path-utils-darwin.c
index 067cbe6990a..13807f58e95 100644
--- a/compat/fsmonitor/fsm-path-utils-darwin.c
+++ b/compat/fsmonitor/fsm-path-utils-darwin.c
@@ -1,5 +1,8 @@
 #include "fsmonitor.h"
 #include "fsmonitor-path-utils.h"
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
 #include <sys/param.h>
 #include <sys/mount.h>
 
@@ -38,3 +41,92 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * Scan the root directory for synthetic firmlinks that when resolved
+ * are a prefix of the path, stopping at the first one found.
+ *
+ * Some information about firmlinks and synthetic firmlinks:
+ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
+ *
+ * macOS no longer allows symlinks in the root directory; any link found
+ * there is therefore a synthetic firmlink.
+ *
+ * If this function gets called often, will want to cache all the firmlink
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
+ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	DIR * dir;
+	int read;
+	int retval;
+	struct dirent *de;
+	struct strbuf alias;
+	struct strbuf points_to;
+
+	retval = 0;
+	dir = opendir("/");
+	if (!dir)
+		return -1;
+
+	strbuf_init(&alias, 256);
+	strbuf_init(&points_to, MAXPATHLEN);
+
+	while ((de = readdir(dir)) != NULL) {
+		strbuf_reset(&alias);
+		strbuf_addch(&alias, '/');
+		strbuf_add(&alias, de->d_name, strlen(de->d_name));
+
+		read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
+		if (read > 0) {
+			strbuf_setlen(&points_to, read);
+			if ((strncmp(points_to.buf, path, points_to.len) == 0)
+				&& path[points_to.len] == '/') {
+				info->alias = strbuf_detach(&alias, NULL);
+				info->points_to = strbuf_detach(&points_to, NULL);
+				trace_printf_key(&trace_fsmonitor,
+					"Found alias for '%s' : '%s' -> '%s'",
+					path, info->alias, info->points_to);
+				retval = 0;
+				goto done;
+			}
+		} else if (errno != EINVAL) { /* Something other than not a link */
+			trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
+			retval = -1;
+			goto done;
+		}
+	}
+
+	done:
+	closedir(dir);
+	strbuf_release(&alias);
+	strbuf_release(&points_to);
+	return retval;
+}
+
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	int len = info->alias ? strlen(info->alias) : 0;
+
+	if (!len)
+		return NULL;
+
+	if ((strncmp(info->alias, path, len) == 0)
+		&& path[len] == '/') {
+		struct strbuf tmp;
+		const char *remainder = path + len;
+		int ptr_len = strlen(info->points_to);
+		int rem_len = strlen(remainder);
+
+		strbuf_init(&tmp, ptr_len + rem_len);
+		strbuf_add(&tmp, info->points_to, ptr_len);
+		strbuf_add(&tmp, remainder, rem_len);
+		return strbuf_detach(&tmp, NULL);
+	}
+
+	return NULL;
+}
diff --git a/compat/fsmonitor/fsm-path-utils-win32.c b/compat/fsmonitor/fsm-path-utils-win32.c
index a90b8f7925b..0d95bbb416f 100644
--- a/compat/fsmonitor/fsm-path-utils-win32.c
+++ b/compat/fsmonitor/fsm-path-utils-win32.c
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)
 		return -1;
 	return fs.is_remote;
 }
+
+/*
+ * No-op for now.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
+	return 0;
+}
+
+/*
+ * No-op for now.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info)
+{
+	return NULL;
+}
diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h
index 2102a5c9ff5..e24838f9a86 100644
--- a/fsmonitor--daemon.h
+++ b/fsmonitor--daemon.h
@@ -8,6 +8,7 @@
 #include "run-command.h"
 #include "simple-ipc.h"
 #include "thread-utils.h"
+#include "fsmonitor-path-utils.h"
 
 struct fsmonitor_batch;
 struct fsmonitor_token_data;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {
 
 	struct strbuf path_worktree_watch;
 	struct strbuf path_gitdir_watch;
+	struct alias_info alias;
 	int nr_paths_watching;
 
 	struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {
 
 	struct ipc_server_data *ipc_server_data;
 	struct strbuf path_ipc;
+
 };
 
 /*
diff --git a/fsmonitor-path-utils.h b/fsmonitor-path-utils.h
index e48592887e7..50ef37e57bb 100644
--- a/fsmonitor-path-utils.h
+++ b/fsmonitor-path-utils.h
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
@@ -1,6 +1,14 @@
 #ifndef FSM_PATH_UTILS_H
 #define FSM_PATH_UTILS_H
 
+#include "strbuf.h"
+
+struct alias_info
+{
+	char *alias;
+	char *points_to;
+};
+
 struct fs_info {
 	int is_remote;
 	char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
  */
 int fsmonitor__is_fs_remote(const char *path);
 
+/*
+ * Get the alias in given path, if any.
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
+ * If an alias is found, info.alias and info.points_to are set to the
+ * found mapping.
+ *
+ * Returns -1 on error, 0 otherwise.
+ *
+ * The caller owns the storage that is occupied by set info.alias and
+ * info.points_to and is responsible for releasing it with `free(3)`
+ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
+/*
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
+ *
+ * The caller owns the storage that the returned string occupies and
+ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+	const struct alias_info *info);
+
+
 #endif
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v9 5/6] fsmonitor: check for compatability before communicating with fsmonitor
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                                   ` (3 preceding siblings ...)
  2022-09-19 19:37                 ` [PATCH v9 4/6] fsmonitor: deal with synthetic firmlinks on macOS Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-19 19:37                 ` [PATCH v9 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
  2022-09-20 20:33                 ` [PATCH v10 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

If fsmonitor is not in a compatible state, die with an appropriate error
messge.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 fsmonitor-settings.c | 2 ++
 fsmonitor.c          | 4 ++++
 2 files changed, 6 insertions(+)

diff --git a/fsmonitor-settings.c b/fsmonitor-settings.c
index 531a1b6f956..aaa204e0352 100644
--- a/fsmonitor-settings.c
+++ b/fsmonitor-settings.c
@@ -247,6 +247,8 @@ char *fsm_settings__get_incompatible_msg(const struct repository *r,
 {
 	struct strbuf msg = STRBUF_INIT;
 
+	strbuf_add(&msg, "fsmonitor: ", strlen("fsmonitor: "));
+
 	switch (reason) {
 	case FSMONITOR_REASON_UNTESTED:
 	case FSMONITOR_REASON_OK:
diff --git a/fsmonitor.c b/fsmonitor.c
index 57d6a483bee..43d580132fb 100644
--- a/fsmonitor.c
+++ b/fsmonitor.c
@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)
 	int is_trivial = 0;
 	struct repository *r = istate->repo ? istate->repo : the_repository;
 	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
+	enum fsmonitor_reason reason = fsm_settings__get_reason(r);
+
+	if (reason > FSMONITOR_REASON_OK)
+		die("%s", fsm_settings__get_incompatible_msg(r, reason));
 
 	if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
 	    istate->fsmonitor_has_run_once)
-- 
@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)
 	int is_trivial = 0;
 	struct repository *r = istate->repo ? istate->repo : the_repository;
 	enum fsmonitor_mode fsm_mode = fsm_settings__get_mode(r);
+	enum fsmonitor_reason reason = fsm_settings__get_reason(r);
+
+	if (reason > FSMONITOR_REASON_OK)
+		die("%s", fsm_settings__get_incompatible_msg(r, reason));
 
 	if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
 	    istate->fsmonitor_has_run_once)
-- 
gitgitgadget


^ permalink raw reply related	[flat|nested] 157+ messages in thread

* [PATCH v9 6/6] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                                   ` (4 preceding siblings ...)
  2022-09-19 19:37                 ` [PATCH v9 5/6] fsmonitor: check for compatability before communicating with fsmonitor Eric DeCosta via GitGitGadget
@ 2022-09-19 19:37                 ` Eric DeCosta via GitGitGadget
  2022-09-20 20:33                 ` [PATCH v10 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
  6 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-19 19:37 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta, Eric DeCosta

From: Eric DeCosta <edecosta@mathworks.com>

Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
Call-out experimental nature of 'fsmonitor.allowRemote' and limited file
system support for 'fsmonitor.socketDir'.

Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
---
 Documentation/git-fsmonitor--daemon.txt | 48 +++++++++++++++++++++++--
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt
index cc142fb8612..6ad3e518ae0 100644
--- a/Documentation/git-fsmonitor--daemon.txt
+++ b/Documentation/git-fsmonitor--daemon.txt
@@ -3,7 +3,7 @@ git-fsmonitor{litdd}daemon(1)
 
 NAME
 ----
-git-fsmonitor--daemon - A Built-in File System Monitor
+git-fsmonitor--daemon - A Built-in Filesystem Monitor
 
 SYNOPSIS
 --------
@@ -17,7 +17,7 @@ DESCRIPTION
 -----------
 
 A daemon to watch the working directory for file and directory
-changes using platform-specific file system notification facilities.
+changes using platform-specific filesystem notification facilities.
 
 This daemon communicates directly with commands like `git status`
 using the link:technical/api-simple-ipc.html[simple IPC] interface
@@ -63,13 +63,55 @@ CAVEATS
 -------
 
 The fsmonitor daemon does not currently know about submodules and does
-not know to filter out file system events that happen within a
+not know to filter out filesystem events that happen within a
 submodule.  If fsmonitor daemon is watching a super repo and a file is
 modified within the working directory of a submodule, it will report
 the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this may be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositories and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a
+special type of file -- which is supported by native Mac OS filesystems,
+but not on network-mounted filesystems, NTFS, or FAT32.  Other filesystems
+may or may not have the needed support; the fsmonitor daemon is not guaranteed
+to work with these filesystems and such use is considered experimental.
+
+By default, the socket is created in the `.git` directory, however, if the
+`.git` directory is on a network-mounted filesystem, it will be instead be
+created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a
+network-mounted filesystem in which case you must set the configuration
+variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native
+filesystem in which to create the socket file.
+
+If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`)
+is on a native Mac OS file filesystem the fsmonitor daemon will report an
+error that will cause the daemon and the currently running command to exit.
+
+CONFIGURATION
+-------------
+
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+`fsmonitor.allowRemote`::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+`fsmonitor.socketDir`::
+    This Mac OS-specific option, if set, specifies the directory in
+    which to create the Unix domain socket used for communication
+    between fsmonitor and various Git commands. The directory must
+    reside on a native Mac OS filesystem as discussed above.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
@@ -3,7 +3,7 @@ git-fsmonitor{litdd}daemon(1)
 
 NAME
 ----
-git-fsmonitor--daemon - A Built-in File System Monitor
+git-fsmonitor--daemon - A Built-in Filesystem Monitor
 
 SYNOPSIS
 --------
@@ -17,7 +17,7 @@ DESCRIPTION
 -----------
 
 A daemon to watch the working directory for file and directory
-changes using platform-specific file system notification facilities.
+changes using platform-specific filesystem notification facilities.
 
 This daemon communicates directly with commands like `git status`
 using the link:technical/api-simple-ipc.html[simple IPC] interface
@@ -63,13 +63,55 @@ CAVEATS
 -------
 
 The fsmonitor daemon does not currently know about submodules and does
-not know to filter out file system events that happen within a
+not know to filter out filesystem events that happen within a
 submodule.  If fsmonitor daemon is watching a super repo and a file is
 modified within the working directory of a submodule, it will report
 the change (as happening against the super repo).  However, the client
 will properly ignore these extra events, so performance may be affected
 but it will not cause an incorrect result.
 
+By default, the fsmonitor daemon refuses to work against network-mounted
+repositories; this may be overridden by setting `fsmonitor.allowRemote` to
+`true`. Note, however, that the fsmonitor daemon is not guaranteed to work
+correctly with all network-mounted repositories and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
+commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a
+special type of file -- which is supported by native Mac OS filesystems,
+but not on network-mounted filesystems, NTFS, or FAT32.  Other filesystems
+may or may not have the needed support; the fsmonitor daemon is not guaranteed
+to work with these filesystems and such use is considered experimental.
+
+By default, the socket is created in the `.git` directory, however, if the
+`.git` directory is on a network-mounted filesystem, it will be instead be
+created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a
+network-mounted filesystem in which case you must set the configuration
+variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native
+filesystem in which to create the socket file.
+
+If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`)
+is on a native Mac OS file filesystem the fsmonitor daemon will report an
+error that will cause the daemon and the currently running command to exit.
+
+CONFIGURATION
+-------------
+
+When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
+the fsmonitor daemon will pay attention to the following configuration
+variables:
+
+`fsmonitor.allowRemote`::
+	By default, the daemon refuses to work against network-mounted
+	repositories. Setting `fsmonitor.allowRemote` to `true` overrides
+	this behavior.
+
+`fsmonitor.socketDir`::
+    This Mac OS-specific option, if set, specifies the directory in
+    which to create the Unix domain socket used for communication
+    between fsmonitor and various Git commands. The directory must
+    reside on a native Mac OS filesystem as discussed above.
+
 GIT
 ---
 Part of the linkgit:git[1] suite
-- 
gitgitgadget

^ permalink raw reply related	[flat|nested] 157+ messages in thread

* Re: [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 19:37                 ` [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote Eric DeCosta via GitGitGadget
@ 2022-09-19 19:57                   ` Eric Sunshine
  0 siblings, 0 replies; 157+ messages in thread
From: Eric Sunshine @ 2022-09-19 19:57 UTC (permalink / raw)
  To: Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

On Mon, Sep 19, 2022 at 3:38 PM Eric DeCosta via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> If the .git directory is on a remote file system, create the socket
> file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
>
> Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> ---
> diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c
> @@ -0,0 +1,48 @@
> +const char *fsmonitor_ipc__get_path(struct repository *r)
> +{
> +       char *sock_dir;
> +
> +       repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
> +
> +       /* Create the socket file in either socketDir or $HOME */
> +       if (sock_dir && *sock_dir) {
> +               strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
> +                                       sock_dir, hash_to_hex(hash));
> +               free(sock_dir);
> +       } else {
> +               strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
> +       }

This is still leaking `sock_dir`, isn't it, if
repo_config_get_string() returns it as a zero-length string?

Probably want to move free() below the entire conditional.

> diff --git a/compat/fsmonitor/fsm-ipc-win32.c b/compat/fsmonitor/fsm-ipc-win32.c
> @@ -0,0 +1,9 @@
> +#include "config.h"
> +#include "fsmonitor-ipc.h"
> +
> +const char *fsmonitor_ipc__get_path(struct repository *r) {
> +       static char *ret;
> +       if (!ret)
> +               ret = git_pathdup("fsmonitor--daemon.ipc");
> +       return ret;
> +}
> \ No newline at end of file

Still an incomplete line.

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 17:49                       ` Junio C Hamano
@ 2022-09-19 23:51                         ` Eric DeCosta
  2022-09-20 14:35                           ` Jeff Hostetler
  0 siblings, 1 reply; 157+ messages in thread
From: Eric DeCosta @ 2022-09-19 23:51 UTC (permalink / raw)
  To: Junio C Hamano, Jeff Hostetler
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Junio C Hamano <gitster@pobox.com>
> Sent: Monday, September 19, 2022 1:49 PM
> To: Jeff Hostetler <git@jeffhostetler.com>
> Cc: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org; Eric Sunshine <sunshine@sunshineco.com>; Torsten
> Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
> Johannes Schindelin <Johannes.Schindelin@gmx.de>; Eric DeCosta
> <edecosta@mathworks.com>
> Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is
> remote
> 
> Jeff Hostetler <git@jeffhostetler.com> writes:
> 
> > Aren't we in the middle of a transition from always using the global
> > "the_repository" to a passed "r" variable?
> > We're getting closer to being able to hide the the global symbol, but
> > we're not there yet, right?
> 
> We may still have code that works ONLY on the_repository, but letting a
> function take "r" and lettin it ignore is worse than leaving it explicitly limited
> to the_repository only, no?
> 
> > I'm thinking that at as long as the global exists, we are not safe to
> > have multiple "struct repository" instances, right?
> 
> By itself, Not at all.  It is the code like I am criticizing that makes it unsafe.
> 
> I do not mind adding
> 
> 	if (!r)
> 		BUG(...);
> 
> at the place you have the "sweep it under the rug" band-aid, though.

Appreciate all the insights and comments. Where are we landing with this? Very close to the finish line and I'd like to be able to push these changes over that line.

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options
  2022-09-17  6:08                 ` Eric Sunshine
@ 2022-09-19 23:55                   ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-19 23:55 UTC (permalink / raw)
  To: Eric Sunshine, Eric DeCosta via GitGitGadget
  Cc: Git List, Jeff Hostetler, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Eric Sunshine <sunshine@sunshineco.com>
> Sent: Saturday, September 17, 2022 2:09 AM
> To: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>
> Cc: Git List <git@vger.kernel.org>; Jeff Hostetler <git@jeffhostetler.com>;
> Torsten Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
> Johannes Schindelin <Johannes.Schindelin@gmx.de>; Eric DeCosta
> <edecosta@mathworks.com>
> Subject: Re: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote
> and socketDir options
> 
> On Fri, Sep 16, 2022 at 9:12 PM Eric DeCosta via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
> > Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'.
> > Call-out experimental nature of 'fsmonitor.allowRemote' and limited
> > file system support for 'fsmonitor.socketDir'.
> >
> > Signed-off-by: Eric DeCosta <edecosta@mathworks.com>
> > ---
> > diff --git a/Documentation/git-fsmonitor--daemon.txt
> > b/Documentation/git-fsmonitor--daemon.txt
> > @@ -70,6 +70,41 @@ the change (as happening against the super repo).
> > However, the client
> > +By default, the fsmonitor daemon refuses to work against
> > +network-mounted repositories; this my be overridden by setting
> > +`fsmonitor.allowRemote` to `true`. Note, however, that the fsmonitor
> > +daemon is not guaranteed to work correctly with all network-mounted
> > +repositores and such use is considered experimental.
> 
> s/repositores/repositories/
> 
> > +On Mac OS, the inter-process communication (IPC) between various Git
> > +commands and the fsmonitor daemon is done via a Unix domain socket
> (UDS).
> > +Usage of UDS requires the creation of a file which, by default, is
> > +created in the .git directory.  If the fsmonitor daemon detects that
> > +the .git directory
> 
> Typesetting: s/.git/`.git`/g
> 
> > +is on a network-mounted file system, it will create the UDS file in
> > +$HOME.  If
> 
> There's a gap in the explanation as to _why_ fsmonitor won't use the .git
> directory for this file when on a network-mounted filesystem and instead
> chooses $HOME. For the reader who is not well-versed in Unix
> sockets/filesystems, it may be difficult to understand the logic behind this.
> This gap is somewhat filled in a few sentences later, but it makes for
> potentially confusing reading until then.
> 
> Should the reader know the name of the socket file or at least the templated
> form of the name? The first question which popped into my head upon
> reading this was whether it was going to pollute my home directory with
> non-hidden files. If this had mentioned something along the lines of
> "creation of a file named `.git-fsmonitor-*`" or "creation of a hidden file"
> then I would have understood immediately that the file would have been
> hidden.
> 
> > +$HOME itself is on a network-mounted file system or if $HOME is not
> > +the desired
> 
> To be consistent with formatting elsewhere in the Git documentation, let's
> typeset this as `$HOME` (with backticks).
> 
> Aside: The spelling "filesystem" appears almost five times as often as "file
> system" in Git documentation, however, this particular file already uses "file
> system" and does so consistently, so it makes sense to follow suit as you do
> here. Changing to use "filesystem" instead, if such a task is desirable, is
> outside the scope of this patch series.
> 
> > +location for the UDS file, 'fsmonitor.socketDir' may be set to any
> > +valid, local
> 
> For consistency, let's use backticks here, as well: `fsmonitor.socketDir`
> 
> > +directory on a file system with proper support.  Mac OS native file
> > +systems have
> 
> Together with the above comment about a gap in the explanation, I found
> myself scratching my head about what "proper support" meant (when
> pretending to read this as a person not particularly familiar with Unix sockets
> or filesystems).
> 
> Also, although this explains how to work around the case when $HOME is
> itself network-mounted, what happens if $HOME is network-mounted and
> the user does not set `fsmonitor.socketDir`? Does it error out? Does it simply
> misbehave in some way? Should it error out? (I would think
> "yes".)
> 
> > +the required support.  File systems known to lack support include
> > +FAT32 and NTFS.  Other file systems may or many not have the needed
> > +support; the fsmonitor
> 
> s/many/may/
> 
> > +daemon is not guaranteed to work with these file systems and such use
> > +is considered experimental.
> 
> Taking the above comments into account, here's my attempt at a rewrite:
> 
>     On Mac OS, the inter-process communication (IPC) between various
>     Git commands and the fsmonitor daemon occurs via a Unix domain
>     socket (UDS) -- a special type of file -- which is supported by
>     the native Mac OS filesystems but not by network-mounted
>     filesystems, NTFS or FAT32.  Other file systems may or many not
>     have the needed support; the fsmonitor daemon is not guaranteed to
>     work with these file systems and such use is considered
>     experimental.
> 
>     By default, the socket is created in the `.git` directory,
>     however, if the `.git` directory is on a network-mounted file
>     system, it will instead be created at `$HOME/.git-fsmonitor-*`
>     unless `$HOME` itself is on a network-mounted file system, in
>     which case you must set the configuration variable
>     `fsmonitor.socketDir` to the path of a directory on a Mac OS
>     native filesystem in which to create the socket file.
> 
> > +CONFIGURATION
> > +-------------
> > +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1])
> > +the fsmonitor daemon will pay attention to the following
> > +configuration
> > +variables:
> 
> We probably want a blank line after the header underline and before this
> paragraph.
> 
> > +fsmonitor.allowRemote::
> > +       By default, the daemon refuses to work against network-mounted
> > +       repositories. Setting `fsmonitor.allowRemote` to `true` overrides
> > +       this behavior.
> > +
> > +fsmonitor.socketDir::
> > +       This option is only used by the Mac OS implementation of the
> fsmonitor
> > +       daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local
> > +       directory on a file system that can support Unix domain sockets (UDS).
> 
> Typeset with backticks: `fsmonitor.socketDir`
> 
> The word "valid" seems unnecessary. A possible rewrite:
> 
>     This Mac OS-specific option, if set, specifies the directory in
>     which to create the Unix domain socket used for communication
>     between fsmonitor and various Git commands. The directory must
>     reside on a native Mac OS filesystem as discussed above.

The way you've re-written things is so much clearer! Much appreciated.

-Eric

^ permalink raw reply	[flat|nested] 157+ messages in thread

* Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-19 23:51                         ` Eric DeCosta
@ 2022-09-20 14:35                           ` Jeff Hostetler
  2022-09-20 15:49                             ` Eric DeCosta
  0 siblings, 1 reply; 157+ messages in thread
From: Jeff Hostetler @ 2022-09-20 14:35 UTC (permalink / raw)
  To: Eric DeCosta, Junio C Hamano
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



On 9/19/22 7:51 PM, Eric DeCosta wrote:
> 
> 
>> -----Original Message-----
>> From: Junio C Hamano <gitster@pobox.com>
>> Sent: Monday, September 19, 2022 1:49 PM
>> To: Jeff Hostetler <git@jeffhostetler.com>
>> Cc: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
>> git@vger.kernel.org; Eric Sunshine <sunshine@sunshineco.com>; Torsten
>> Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
>> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
>> Johannes Schindelin <Johannes.Schindelin@gmx.de>; Eric DeCosta
>> <edecosta@mathworks.com>
>> Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is
>> remote
>>
>> Jeff Hostetler <git@jeffhostetler.com> writes:
>>
>>> Aren't we in the middle of a transition from always using the global
>>> "the_repository" to a passed "r" variable?
>>> We're getting closer to being able to hide the the global symbol, but
>>> we're not there yet, right?
>>
>> We may still have code that works ONLY on the_repository, but letting a
>> function take "r" and lettin it ignore is worse than leaving it explicitly limited
>> to the_repository only, no?
>>
>>> I'm thinking that at as long as the global exists, we are not safe to
>>> have multiple "struct repository" instances, right?
>>
>> By itself, Not at all.  It is the code like I am criticizing that makes it unsafe.
>>
>> I do not mind adding
>>
>> 	if (!r)
>> 		BUG(...);
>>
>> at the place you have the "sweep it under the rug" band-aid, though.
> 
> Appreciate all the insights and comments. Where are we landing with this? Very close to the finish line and I'd like to be able to push these changes over that line.
> 
> -Eric
> 

I'm OK doing it either way.  Junio seems to prefer the BUG() version,
so let's go with that.  That lets us make progress on getting rid of
direct references to "the_repository".

Jeff

^ permalink raw reply	[flat|nested] 157+ messages in thread

* RE: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
  2022-09-20 14:35                           ` Jeff Hostetler
@ 2022-09-20 15:49                             ` Eric DeCosta
  0 siblings, 0 replies; 157+ messages in thread
From: Eric DeCosta @ 2022-09-20 15:49 UTC (permalink / raw)
  To: Jeff Hostetler, Junio C Hamano
  Cc: Eric DeCosta via GitGitGadget, git, Eric Sunshine,
	Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin



> -----Original Message-----
> From: Jeff Hostetler <git@jeffhostetler.com>
> Sent: Tuesday, September 20, 2022 10:36 AM
> To: Eric DeCosta <edecosta@mathworks.com>; Junio C Hamano
> <gitster@pobox.com>
> Cc: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> git@vger.kernel.org; Eric Sunshine <sunshine@sunshineco.com>; Torsten
> Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
> Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is
> remote
> 
> 
> 
> On 9/19/22 7:51 PM, Eric DeCosta wrote:
> >
> >
> >> -----Original Message-----
> >> From: Junio C Hamano <gitster@pobox.com>
> >> Sent: Monday, September 19, 2022 1:49 PM
> >> To: Jeff Hostetler <git@jeffhostetler.com>
> >> Cc: Eric DeCosta via GitGitGadget <gitgitgadget@gmail.com>;
> >> git@vger.kernel.org; Eric Sunshine <sunshine@sunshineco.com>; Torsten
> >> Bögershausen <tboegi@web.de>; Ævar Arnfjörð Bjarmason
> >> <avarab@gmail.com>; Ramsay Jones <ramsay@ramsayjones.plus.com>;
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de>; Eric DeCosta
> >> <edecosta@mathworks.com>
> >> Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git
> >> directory is remote
> >>
> >> Jeff Hostetler <git@jeffhostetler.com> writes:
> >>
> >>> Aren't we in the middle of a transition from always using the global
> >>> "the_repository" to a passed "r" variable?
> >>> We're getting closer to being able to hide the the global symbol,
> >>> but we're not there yet, right?
> >>
> >> We may still have code that works ONLY on the_repository, but letting
> >> a function take "r" and lettin it ignore is worse than leaving it
> >> explicitly limited to the_repository only, no?
> >>
> >>> I'm thinking that at as long as the global exists, we are not safe
> >>> to have multiple "struct repository" instances, right?
> >>
> >> By itself, Not at all.  It is the code like I am criticizing that makes it unsafe.
> >>
> >> I do not mind adding
> >>
> >> 	if (!r)
> >> 		BUG(...);
> >>
> >> at the place you have the "sweep it under the rug" band-aid, though.
> >
> > Appreciate all the insights and comments. Where are we landing with this?
> Very close to the finish line and I'd like to be able to push these changes over
> that line.
> >
> > -Eric
> >
> 
> I'm OK doing it either way.  Junio seems to prefer the BUG() version, so let's
> go with that.  That lets us make progress on getting rid of direct references to
> "the_repository".
> 
> Jeff

Sounds like a plan!

-Eric


^ permalink raw reply	[flat|nested] 157+ messages in thread

* [PATCH v10 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos
  2022-09-19 19:37               ` [PATCH v9 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted repos Eric DeCosta via GitGitGadget
                                   ` (5 preceding siblings ...)
  2022-09-19 19:37                 ` [PATCH v9 6/6] fsmonitor: add documentation for allowRemote and socketDir options Eric DeCosta via GitGitGadget
@ 2022-09-20 20:33                 ` Eric DeCosta via GitGitGadget
  2022-09-20 20:33                   ` [PATCH v10 1/6] fsmonitor: refactor filesystem checks to common interface Eric DeCosta via GitGitGadget
                                     ` (6 more replies)
  6 siblings, 7 replies; 157+ messages in thread
From: Eric DeCosta via GitGitGadget @ 2022-09-20 20:33 UTC (permalink / raw)
  To: git
  Cc: Jeff Hostetler, Eric Sunshine, Torsten Bögershausen,
	Ævar Arnfjörð Bjarmason, Ramsay Jones,
	Johannes Schindelin, Eric DeCosta

Follow-on to the work done to allow Windows to work against network-mounted
repos for macOS.

Have macOS take advantage of the same configuration option,
'fsmonitor.allowRemote' that was introduced for Windows. Setting this option
to true will override the default behavior (erroring-out) when a
network-mounted repo is detected by fsmonitor.

The added wrinkle being that the Unix domain socket (UDS) file used for IPC
cannot be created in a network location; instead $HOME is used if the
default location is on the network. The user may, optionally, set the
'fsmonitor.socketDir' configuration option to a valid, local directory if
$HOME itself is on the network or is simply not the desired location for the
UDS file.

An additional issue is that for mount points in the root directory, FSEvents
does not report a path that matches the worktree directory due to the
introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths
to the worktree directory by interrogating the root filesystem for synthetic
firmlinks and using that information to translate the path.

v10 differs from v9:

 * incorporates code review feedback
 * improves error messaging for incompatible socket directory

v9 differs from v8:

 * incorporates code review feedback
 * check for incompatibility before communicating with fsmonitor

v8 differs from v7:

 * incorporates code review feedback
 * gets the rebase right

v7 differs from v6:

 * incorporates code review feedback

v6 differs from v5:

 * incorporates earlier, Windows-specific changes that have not made it back
   yet to the master branch
 * incorporates code review feedback
 * adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'

v5 differs significantly from earlier versions:

 * redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
   such that these options are no longer added to the settings data
   structure but are rather read from config at point of use
 * refactoring of code for handling platform-specific file system checks via
   a common interface to avoid platform #ifdef in IPC code and be in-model
   with other platform-specific fsmonitor code
 * dealing with 'synthetic firmlinks' on macOS

Eric DeCosta (6):
  fsmonitor: refactor filesystem checks to common interface
  fsmonitor: relocate socket file if .git directory is remote
  fsmonitor: avoid socket location check if using hook
  fsmonitor: deal with synthetic firmlinks on macOS
  fsmonitor: check for compatability before communicating with fsmonitor
  fsmonitor: add documentation for allowRemote and socketDir options

 Documentation/git-fsmonitor--daemon.txt  |  48 ++++++-
 Makefile                                 |   2 +
 builtin/fsmonitor--daemon.c              |  11 +-
 compat/fsmonitor/fsm-ipc-darwin.c        |  51 +++++++
 compat/fsmonitor/fsm-ipc-win32.c         |   9 ++
 compat/fsmonitor/fsm-listen-darwin.c     |   6 +-
 compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++
 compat/fsmonitor/fsm-path-utils-win32.c  | 145 +++++++++++++++++++
 compat/fsmonitor/fsm-settings-darwin.c   |  72 +++-------
 compat/fsmonitor/fsm-settings-win32.c    | 174 +----------------------
 contrib/buildsystems/CMakeLists.txt      |   4 +
 fsmonitor--daemon.h                      |   3 +
 fsmonitor-ipc.c                          |  18 ++-
 fsmonitor-ipc.h                          |   4 +-
 fsmonitor-path-utils.h                   |  59 ++++++++
 fsmonitor-settings.c                     |  68 ++++++++-
 fsmonitor-settings.h                     |   4 +-
 fsmonitor.c                              |   4 +
 18 files changed, 569 insertions(+), 245 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c
 create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c
 create mode 100644 fsmonitor-path-utils.h


base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v10
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v10
Pull-Request: https://github.com/gitgitgadget/git/pull/1326

Range-diff vs v9:

 1:  155a6890806 = 1:  155a6890806 fsmonitor: refactor filesystem checks to common interface
 2:  68709d788d5 ! 2:  dbe113abb87 fsmonitor: relocate socket file if .git directory is remote
     @@ compat/fsmonitor/fsm-ipc-darwin.c (new)
      +{
      +	static const char *ipc_path;
      +	SHA_CTX sha1ctx;
     -+	char *sock_dir;
     ++	char *sock_dir = NULL;
      +	struct strbuf ipc_file = STRBUF_INIT;
      +	unsigned char hash[SHA_DIGEST_LENGTH];
      +