From: "Jeff Hostetler via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: "Patrick Steinhardt" <ps@pks.im>,
"Jeff Hostetler" <git@jeffhostetler.com>,
"Torsten Bögershausen" <tboegi@web.de>,
"Jeff Hostetler" <jeffhostetler@github.com>,
"Jeff Hostetler" <jeffhostetler@github.com>
Subject: [PATCH v3 14/14] fsmonitor: support case-insensitive events
Date: Mon, 26 Feb 2024 21:39:25 +0000 [thread overview]
Message-ID: <ec036c04d1b003385683f0ed1dec80cc3bbfb813.1708983566.git.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.1662.v3.git.1708983565.gitgitgadget@gmail.com>
From: Jeff Hostetler <jeffhostetler@github.com>
Teach fsmonitor_refresh_callback() to handle case-insensitive
lookups if case-sensitive lookups fail on case-insensitive systems.
This can cause 'git status' to report stale status for files if there
are case issues/errors in the worktree.
The FSMonitor daemon sends FSEvents using the observed spelling
of each pathname. On case-insensitive file systems this may be
different than the expected case spelling.
The existing code uses index_name_pos() to find the cache-entry for
the pathname in the FSEvent and clear the CE_FSMONITOR_VALID bit so
that the worktree scan/index refresh will revisit and revalidate the
path.
On a case-insensitive file system, the exact match lookup may fail
to find the associated cache-entry. This causes status to think that
the cached CE flags are correct and skip over the file.
Update event handling to optionally use the name-hash and dir-name-hash
if necessary.
Also update t7527 to convert the "test_expect_failure" to "_success"
now that we have fixed the bug.
Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
---
fsmonitor.c | 121 +++++++++++++++++++++++++++++++++++
t/t7527-builtin-fsmonitor.sh | 26 +++++---
2 files changed, 137 insertions(+), 10 deletions(-)
diff --git a/fsmonitor.c b/fsmonitor.c
index 3c87449be87..2b17d60bbbe 100644
--- a/fsmonitor.c
+++ b/fsmonitor.c
@@ -5,6 +5,7 @@
#include "ewah/ewok.h"
#include "fsmonitor.h"
#include "fsmonitor-ipc.h"
+#include "name-hash.h"
#include "run-command.h"
#include "strbuf.h"
#include "trace2.h"
@@ -202,6 +203,113 @@ static void invalidate_ce_fsm(struct cache_entry *ce)
static size_t handle_path_with_trailing_slash(
struct index_state *istate, const char *name, int pos);
+/*
+ * Use the name-hash to do a case-insensitive cache-entry lookup with
+ * the pathname and invalidate the cache-entry.
+ *
+ * Returns the number of cache-entries that we invalidated.
+ */
+static size_t handle_using_name_hash_icase(
+ struct index_state *istate, const char *name)
+{
+ struct cache_entry *ce = NULL;
+
+ ce = index_file_exists(istate, name, strlen(name), 1);
+ if (!ce)
+ return 0;
+
+ /*
+ * A case-insensitive search in the name-hash using the
+ * observed pathname found a cache-entry, so the observed path
+ * is case-incorrect. Invalidate the cache-entry and use the
+ * correct spelling from the cache-entry to invalidate the
+ * untracked-cache. Since we now have sparse-directories in
+ * the index, the observed pathname may represent a regular
+ * file or a sparse-index directory.
+ *
+ * Note that we should not have seen FSEvents for a
+ * sparse-index directory, but we handle it just in case.
+ *
+ * Either way, we know that there are not any cache-entries for
+ * children inside the cone of the directory, so we don't need to
+ * do the usual scan.
+ */
+ trace_printf_key(&trace_fsmonitor,
+ "fsmonitor_refresh_callback MAP: '%s' '%s'",
+ name, ce->name);
+
+ /*
+ * NEEDSWORK: We used the name-hash to find the correct
+ * case-spelling of the pathname in the cache-entry[], so
+ * technically this is a tracked file or a sparse-directory.
+ * It should not have any entries in the untracked-cache, so
+ * we should not need to use the case-corrected spelling to
+ * invalidate the the untracked-cache. So we may not need to
+ * do this. For now, I'm going to be conservative and always
+ * do it; we can revisit this later.
+ */
+ untracked_cache_invalidate_trimmed_path(istate, ce->name, 0);
+
+ invalidate_ce_fsm(ce);
+ return 1;
+}
+
+/*
+ * Use the dir-name-hash to find the correct-case spelling of the
+ * directory. Use the canonical spelling to invalidate all of the
+ * cache-entries within the matching cone.
+ *
+ * Returns the number of cache-entries that we invalidated.
+ */
+static size_t handle_using_dir_name_hash_icase(
+ struct index_state *istate, const char *name)
+{
+ struct strbuf canonical_path = STRBUF_INIT;
+ int pos;
+ size_t len = strlen(name);
+ size_t nr_in_cone;
+
+ if (name[len - 1] == '/')
+ len--;
+
+ if (!index_dir_find(istate, name, len, &canonical_path))
+ return 0; /* name is untracked */
+
+ if (!memcmp(name, canonical_path.buf, canonical_path.len)) {
+ strbuf_release(&canonical_path);
+ /*
+ * NEEDSWORK: Our caller already tried an exact match
+ * and failed to find one. They called us to do an
+ * ICASE match, so we should never get an exact match,
+ * so we could promote this to a BUG() here if we
+ * wanted to. It doesn't hurt anything to just return
+ * 0 and go on because we should never get here. Or we
+ * could just get rid of the memcmp() and this "if"
+ * clause completely.
+ */
+ BUG("handle_using_dir_name_hash_icase(%s) did not exact match",
+ name);
+ }
+
+ trace_printf_key(&trace_fsmonitor,
+ "fsmonitor_refresh_callback MAP: '%s' '%s'",
+ name, canonical_path.buf);
+
+ /*
+ * The dir-name-hash only tells us the corrected spelling of
+ * the prefix. We have to use this canonical path to do a
+ * lookup in the cache-entry array so that we repeat the
+ * original search using the case-corrected spelling.
+ */
+ strbuf_addch(&canonical_path, '/');
+ pos = index_name_pos(istate, canonical_path.buf,
+ canonical_path.len);
+ nr_in_cone = handle_path_with_trailing_slash(
+ istate, canonical_path.buf, pos);
+ strbuf_release(&canonical_path);
+ return nr_in_cone;
+}
+
/*
* The daemon sent an observed pathname without a trailing slash.
* (This is the normal case.) We do not know if it is a tracked or
@@ -335,6 +443,19 @@ static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
else
nr_in_cone = handle_path_without_trailing_slash(istate, name, pos);
+ /*
+ * If we did not find an exact match for this pathname or any
+ * cache-entries with this directory prefix and we're on a
+ * case-insensitive file system, try again using the name-hash
+ * and dir-name-hash.
+ */
+ if (!nr_in_cone && ignore_case) {
+ nr_in_cone = handle_using_name_hash_icase(istate, name);
+ if (!nr_in_cone)
+ nr_in_cone = handle_using_dir_name_hash_icase(
+ istate, name);
+ }
+
if (nr_in_cone)
trace_printf_key(&trace_fsmonitor,
"fsmonitor_refresh_callback CNT: %d",
diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh
index 830f2d9de33..730f3c7f810 100755
--- a/t/t7527-builtin-fsmonitor.sh
+++ b/t/t7527-builtin-fsmonitor.sh
@@ -1051,7 +1051,7 @@ test_expect_success 'split-index and FSMonitor work well together' '
#
# The setup is a little contrived.
#
-test_expect_failure CASE_INSENSITIVE_FS 'fsmonitor subdir case wrong on disk' '
+test_expect_success CASE_INSENSITIVE_FS 'fsmonitor subdir case wrong on disk' '
test_when_finished "stop_daemon_delete_repo subdir_case_wrong" &&
git init subdir_case_wrong &&
@@ -1116,19 +1116,19 @@ test_expect_failure CASE_INSENSITIVE_FS 'fsmonitor subdir case wrong on disk' '
grep -q "dir1/DIR2/dir3/file3.*pos -3" "$PWD/subdir_case_wrong.log1" &&
+ # Verify that we get a mapping event to correct the case.
+ grep -q "MAP:.*dir1/DIR2/dir3/file3.*dir1/dir2/dir3/file3" \
+ "$PWD/subdir_case_wrong.log1" &&
+
# The refresh-callbacks should have caused "git status" to clear
# the CE_FSMONITOR_VALID bit on each of those files and caused
# the worktree scan to visit them and mark them as modified.
grep -q " M AAA" "$PWD/subdir_case_wrong.out" &&
grep -q " M zzz" "$PWD/subdir_case_wrong.out" &&
-
- # Expect Breakage: with the case confusion, the "(pos -3)" causes
- # the client to not clear the CE_FSMONITOR_VALID bit and therefore
- # status will not rescan the file and therefore not report it as dirty.
grep -q " M dir1/dir2/dir3/file3" "$PWD/subdir_case_wrong.out"
'
-test_expect_failure CASE_INSENSITIVE_FS 'fsmonitor file case wrong on disk' '
+test_expect_success CASE_INSENSITIVE_FS 'fsmonitor file case wrong on disk' '
test_when_finished "stop_daemon_delete_repo file_case_wrong" &&
git init file_case_wrong &&
@@ -1242,14 +1242,20 @@ test_expect_failure CASE_INSENSITIVE_FS 'fsmonitor file case wrong on disk' '
GIT_TRACE_FSMONITOR="$PWD/file_case_wrong-try3.log" \
git -C file_case_wrong --no-optional-locks status --short \
>"$PWD/file_case_wrong-try3.out" &&
+
+ # Verify that we get a mapping event to correct the case.
+ grep -q "fsmonitor_refresh_callback MAP:.*dir1/dir2/dir3/FILE-3-A.*dir1/dir2/dir3/file-3-a" \
+ "$PWD/file_case_wrong-try3.log" &&
+ grep -q "fsmonitor_refresh_callback MAP:.*dir1/dir2/dir4/file-4-a.*dir1/dir2/dir4/FILE-4-A" \
+ "$PWD/file_case_wrong-try3.log" &&
+
# FSEvents are in observed case.
grep -q "fsmonitor_refresh_callback.*FILE-3-A.*pos -3" "$PWD/file_case_wrong-try3.log" &&
grep -q "fsmonitor_refresh_callback.*file-4-a.*pos -9" "$PWD/file_case_wrong-try3.log" &&
- # Expect Breakage: with the case confusion, the "(pos-3)" and
- # "(pos -9)" causes the client to not clear the CE_FSMONITOR_VALID
- # bit and therefore status will not rescan the files and therefore
- # not report them as dirty.
+ # The refresh-callbacks should have caused "git status" to clear
+ # the CE_FSMONITOR_VALID bit on each of those files and caused
+ # the worktree scan to visit them and mark them as modified.
grep -q " M dir1/dir2/dir3/file-3-a" "$PWD/file_case_wrong-try3.out" &&
grep -q " M dir1/dir2/dir4/FILE-4-A" "$PWD/file_case_wrong-try3.out"
'
--
gitgitgadget
next prev parent reply other threads:[~2024-02-26 21:41 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-13 20:52 [PATCH 00/12] FSMonitor edge cases on case-insensitive file systems Jeff Hostetler via GitGitGadget
2024-02-13 20:52 ` [PATCH 01/12] sparse-index: pass string length to index_file_exists() Jeff Hostetler via GitGitGadget
2024-02-13 22:07 ` Junio C Hamano
2024-02-20 17:34 ` Jeff Hostetler
2024-02-13 20:52 ` [PATCH 02/12] name-hash: add index_dir_exists2() Jeff Hostetler via GitGitGadget
2024-02-13 21:43 ` Junio C Hamano
2024-02-20 17:38 ` Jeff Hostetler
2024-02-20 19:34 ` Junio C Hamano
2024-02-15 9:31 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 03/12] t7527: add case-insensitve test for FSMonitor Jeff Hostetler via GitGitGadget
2024-02-13 20:52 ` [PATCH 04/12] fsmonitor: refactor refresh callback on directory events Jeff Hostetler via GitGitGadget
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-20 18:54 ` Jeff Hostetler
2024-02-21 12:54 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 05/12] fsmonitor: refactor refresh callback for non-directory events Jeff Hostetler via GitGitGadget
2024-02-14 1:34 ` Junio C Hamano
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 06/12] fsmonitor: clarify handling of directory events in callback Jeff Hostetler via GitGitGadget
2024-02-14 7:47 ` Junio C Hamano
2024-02-20 18:56 ` Jeff Hostetler
2024-02-20 19:24 ` Junio C Hamano
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-20 19:10 ` Jeff Hostetler
2024-02-13 20:52 ` [PATCH 07/12] fsmonitor: refactor untracked-cache invalidation Jeff Hostetler via GitGitGadget
2024-02-14 16:46 ` Junio C Hamano
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 08/12] fsmonitor: support case-insensitive directory events Jeff Hostetler via GitGitGadget
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 09/12] fsmonitor: refactor non-directory callback Jeff Hostetler via GitGitGadget
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 10/12] fsmonitor: support case-insensitive non-directory events Jeff Hostetler via GitGitGadget
2024-02-13 20:52 ` [PATCH 11/12] fsmonitor: refactor bit invalidation in refresh callback Jeff Hostetler via GitGitGadget
2024-02-15 9:32 ` Patrick Steinhardt
2024-02-13 20:52 ` [PATCH 12/12] t7527: update case-insenstive fsmonitor test Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 00/16] FSMonitor edge cases on case-insensitive file systems Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 01/16] name-hash: add index_dir_find() Jeff Hostetler via GitGitGadget
2024-02-23 6:37 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 02/16] t7527: add case-insensitve test for FSMonitor Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 03/16] t7527: temporarily disable case-insensitive tests Jeff Hostetler via GitGitGadget
2024-02-23 8:17 ` Junio C Hamano
2024-02-26 17:12 ` Jeff Hostetler
2024-02-23 3:18 ` [PATCH v2 04/16] fsmonitor: refactor refresh callback on directory events Jeff Hostetler via GitGitGadget
2024-02-23 8:18 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 05/16] fsmonitor: clarify handling of directory events in callback helper Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 06/16] fsmonitor: refactor refresh callback for non-directory events Jeff Hostetler via GitGitGadget
2024-02-23 8:18 ` Junio C Hamano
2024-02-25 12:30 ` Torsten Bögershausen
2024-02-25 17:24 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 07/16] dir: create untracked_cache_invalidate_trimmed_path() Jeff Hostetler via GitGitGadget
2024-02-25 12:35 ` Torsten Bögershausen
2024-02-23 3:18 ` [PATCH v2 08/16] fsmonitor: refactor untracked-cache invalidation Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 09/16] fsmonitor: move untracked invalidation into helper functions Jeff Hostetler via GitGitGadget
2024-02-23 17:36 ` Junio C Hamano
2024-02-26 18:45 ` Jeff Hostetler
2024-02-23 3:18 ` [PATCH v2 10/16] fsmonitor: return invalidated cache-entry count on directory event Jeff Hostetler via GitGitGadget
2024-02-23 3:18 ` [PATCH v2 11/16] fsmonitor: remove custom loop from non-directory path handler Jeff Hostetler via GitGitGadget
2024-02-23 17:47 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 12/16] fsmonitor: return invalided cache-entry count on non-directory event Jeff Hostetler via GitGitGadget
2024-02-23 17:51 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 13/16] fsmonitor: trace the new invalidated cache-entry count Jeff Hostetler via GitGitGadget
2024-02-23 17:53 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 14/16] fsmonitor: support case-insensitive events Jeff Hostetler via GitGitGadget
2024-02-23 18:14 ` Junio C Hamano
2024-02-26 20:41 ` Jeff Hostetler
2024-02-26 21:18 ` Junio C Hamano
2024-02-25 13:10 ` Torsten Bögershausen
2024-02-26 20:47 ` Jeff Hostetler
2024-02-23 3:18 ` [PATCH v2 15/16] fsmonitor: refactor bit invalidation in refresh callback Jeff Hostetler via GitGitGadget
2024-02-23 18:18 ` Junio C Hamano
2024-02-23 3:18 ` [PATCH v2 16/16] t7527: update case-insenstive fsmonitor test Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 00/14] FSMonitor edge cases on case-insensitive file systems Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 01/14] name-hash: add index_dir_find() Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 02/14] t7527: add case-insensitve test for FSMonitor Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 03/14] fsmonitor: refactor refresh callback on directory events Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 04/14] fsmonitor: clarify handling of directory events in callback helper Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 05/14] fsmonitor: refactor refresh callback for non-directory events Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 06/14] dir: create untracked_cache_invalidate_trimmed_path() Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 07/14] fsmonitor: refactor untracked-cache invalidation Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 08/14] fsmonitor: move untracked-cache invalidation into helper functions Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 09/14] fsmonitor: return invalidated cache-entry count on directory event Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 10/14] fsmonitor: remove custom loop from non-directory path handler Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 11/14] fsmonitor: return invalided cache-entry count on non-directory event Jeff Hostetler via GitGitGadget
2024-03-06 12:58 ` Patrick Steinhardt
2024-02-26 21:39 ` [PATCH v3 12/14] fsmonitor: trace the new invalidated cache-entry count Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` [PATCH v3 13/14] fsmonitor: refactor bit invalidation in refresh callback Jeff Hostetler via GitGitGadget
2024-02-26 21:39 ` Jeff Hostetler via GitGitGadget [this message]
2024-03-06 12:58 ` [PATCH v3 14/14] fsmonitor: support case-insensitive events Patrick Steinhardt
2024-02-27 1:40 ` [PATCH v3 00/14] FSMonitor edge cases on case-insensitive file systems Junio C Hamano
2024-03-06 12:58 ` Patrick Steinhardt
2024-03-06 17:09 ` Junio C Hamano
2024-03-06 18:10 ` Jeff Hostetler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ec036c04d1b003385683f0ed1dec80cc3bbfb813.1708983566.git.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=git@jeffhostetler.com \
--cc=git@vger.kernel.org \
--cc=jeffhostetler@github.com \
--cc=ps@pks.im \
--cc=tboegi@web.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).