[PATCH] fsmonitor: only enable it in non-bare repositories

[PATCH] fsmonitor: only enable it in non-bare repositories

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The entire point of the FSMonitor is to monitor the worktree changes in
a more efficient manner than `lstat()`ing all worktree files every time
we refresh the index.

But if there is no worktree, FSMonitor has nothing to monitor.

So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
and we're running in a repository without worktree.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
    fsmonitor: only enable it in non-bare repositories
    
    Since I released Git for Windows v2.31.0, with brief interruption of two
    weeks, I enabled the built-in FSMonitor via my user config, and today
    was the first time I did anything in a bare repository. I was somewhat
    surprised that FSMonitor gave me trouble there, as the FSMonitor does
    not even make sense there...
    
    This patch applies on top of jh/rfc-builtin-fsmonitor (not because it
    fixes a problem in the built-in FSMonitor, the bug existed for a long,
    long time before those patches, but because it would otherwise cause
    merge conflicts with that patch series).

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/942

 config.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/config.c b/config.c
index 53e7dedc60de..fc5e744d81ca 100644
--- a/config.c
+++ b/config.c
@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = 0;
+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
-- 
gitgitgadget

Re: [PATCH] fsmonitor: only enable it in non-bare repositories

[Junio C Hamano]

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The entire point of the FSMonitor is to monitor the worktree changes in
> a more efficient manner than `lstat()`ing all worktree files every time
> we refresh the index.
>
> But if there is no worktree, FSMonitor has nothing to monitor.
>
> So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
> and we're running in a repository without worktree.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>     fsmonitor: only enable it in non-bare repositories
>     
>     Since I released Git for Windows v2.31.0, with brief interruption of two
>     weeks, I enabled the built-in FSMonitor via my user config, and today
>     was the first time I did anything in a bare repository. I was somewhat
>     surprised that FSMonitor gave me trouble there, as the FSMonitor does
>     not even make sense there...
>     
>     This patch applies on top of jh/rfc-builtin-fsmonitor (not because it
>     fixes a problem in the built-in FSMonitor, the bug existed for a long,
>     long time before those patches, but because it would otherwise cause
>     merge conflicts with that patch series).
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/942
>
>  config.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/config.c b/config.c
> index 53e7dedc60de..fc5e744d81ca 100644
> --- a/config.c
> +++ b/config.c
> @@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
>  
>  int repo_config_get_fsmonitor(struct repository *r)
>  {
> +	if (!r->worktree) {
> +		/* FSMonitor makes no sense in bare repositories */
> +		core_fsmonitor = 0;

Use NULL instead of integer 0 to mollify SP.


> +		return 1;
> +	}
> +
>  	if (r->settings.use_builtin_fsmonitor > 0) {
>  		core_fsmonitor = "(built-in daemon)";
>  		return 1;
>
> base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

[PATCH v2] fsmonitor: only enable it in non-bare repositories

[Johannes Schindelin via GitGitGadget]

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The entire point of the FSMonitor is to monitor the worktree changes in
a more efficient manner than `lstat()`ing all worktree files every time
we refresh the index.

But if there is no worktree, FSMonitor has nothing to monitor.

So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
and we're running in a repository without worktree.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
    fsmonitor: only enable it in non-bare repositories
    
    For quite a while now, I run with the built-in FSMonitor via my user
    config. Happily, the only issue I ran into was that FSMonitor tried to
    run in a bare repository the other day. But an FSMonitor makes only
    sense if we have a worktree. So let's disable it automatically in bare
    repositories.
    
    This patch applies near the top of jh/rfc-builtin-fsmonitor. I would
    like to keep it as a separate topic because the built-in FSMonitor did
    not introduce this bug. This bug has been in Git's FSMonitor feature for
    a long, long time.
    
    Changes since v1:
    
     * Using NULL instead of 0 (d'oh!)

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/942

Range-diff vs v1:

 1:  3a93be538d33 ! 1:  95333bbf45d5 fsmonitor: only enable it in non-bare repositories
     @@ config.c: int git_config_get_max_percent_split_change(void)
       {
      +	if (!r->worktree) {
      +		/* FSMonitor makes no sense in bare repositories */
     -+		core_fsmonitor = 0;
     ++		core_fsmonitor = NULL;
      +		return 1;
      +	}
      +


 config.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/config.c b/config.c
index 53e7dedc60de..90ce4c23d3dc 100644
--- a/config.c
+++ b/config.c
@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = NULL;
+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
-- 
gitgitgadget

Re: [PATCH] fsmonitor: only enable it in non-bare repositories

[Ævar Arnfjörð Bjarmason]

On Thu, Apr 29 2021, Johannes Schindelin via GitGitGadget wrote:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> The entire point of the FSMonitor is to monitor the worktree changes in
> a more efficient manner than `lstat()`ing all worktree files every time
> we refresh the index.
>
> But if there is no worktree, FSMonitor has nothing to monitor.
>
> So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
> and we're running in a repository without worktree.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>     fsmonitor: only enable it in non-bare repositories
>     
>     Since I released Git for Windows v2.31.0, with brief interruption of two
>     weeks, I enabled the built-in FSMonitor via my user config, and today
>     was the first time I did anything in a bare repository. I was somewhat
>     surprised that FSMonitor gave me trouble there, as the FSMonitor does
>     not even make sense there...
>     
>     This patch applies on top of jh/rfc-builtin-fsmonitor (not because it
>     fixes a problem in the built-in FSMonitor, the bug existed for a long,
>     long time before those patches, but because it would otherwise cause
>     merge conflicts with that patch series).
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/942
>
>  config.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/config.c b/config.c
> index 53e7dedc60de..fc5e744d81ca 100644
> --- a/config.c
> +++ b/config.c
> @@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
>  
>  int repo_config_get_fsmonitor(struct repository *r)
>  {
> +	if (!r->worktree) {
> +		/* FSMonitor makes no sense in bare repositories */
> +		core_fsmonitor = 0;
> +		return 1;
> +	}
> +
>  	if (r->settings.use_builtin_fsmonitor > 0) {
>  		core_fsmonitor = "(built-in daemon)";
>  		return 1;
>
> base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

This is surely a correct fix for now, but wouldn't it in the future also
be useful to run it in bare repositories e.g. to be able cache lookups
for non-existing loose objects?

Re: [PATCH] fsmonitor: only enable it in non-bare repositories

[Jeff Hostetler]

On 5/3/21 9:58 AM, Ævar Arnfjörð Bjarmason wrote:
> 
> On Thu, Apr 29 2021, Johannes Schindelin via GitGitGadget wrote:
> 
>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>>
>> The entire point of the FSMonitor is to monitor the worktree changes in
>> a more efficient manner than `lstat()`ing all worktree files every time
>> we refresh the index.
>>
>> But if there is no worktree, FSMonitor has nothing to monitor.
>>
>> So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
>> and we're running in a repository without worktree.
>>
>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>> ---
...
>>
>> base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
> 
> This is surely a correct fix for now, but wouldn't it in the future also
> be useful to run it in bare repositories e.g. to be able cache lookups
> for non-existing loose objects?
> 

No, the FSMonitor feature only expects data for paths within the
working directory.  (And is independent of whether the FS change
data is provided by my fsmonitor--daemon or provided by a hook-based
provider, such as Watchman.)  The FSMonitor feature uses that data to
shortcut scans of the working directory.

There is no interaction with the contents of the .git/objects
directory and I'm not sure how that would work.

Jeff

Re: [PATCH] fsmonitor: only enable it in non-bare repositories

[Ævar Arnfjörð Bjarmason]

On Mon, May 03 2021, Jeff Hostetler wrote:

> On 5/3/21 9:58 AM, Ævar Arnfjörð Bjarmason wrote:
>> On Thu, Apr 29 2021, Johannes Schindelin via GitGitGadget wrote:
>> 
>>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>>>
>>> The entire point of the FSMonitor is to monitor the worktree changes in
>>> a more efficient manner than `lstat()`ing all worktree files every time
>>> we refresh the index.
>>>
>>> But if there is no worktree, FSMonitor has nothing to monitor.
>>>
>>> So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
>>> and we're running in a repository without worktree.
>>>
>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>> ---
> ...
>>>
>>> base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
>> This is surely a correct fix for now, but wouldn't it in the future
>> also
>> be useful to run it in bare repositories e.g. to be able cache lookups
>> for non-existing loose objects?
>> 
>
> No, the FSMonitor feature only expects data for paths within the
> working directory.  (And is independent of whether the FS change
> data is provided by my fsmonitor--daemon or provided by a hook-based
> provider, such as Watchman.)  The FSMonitor feature uses that data to
> shortcut scans of the working directory.

Indeed, hence "in the future". I'm not suggesting that it'll do anything
useful by watching anything in the .git directory now, but that it might
be an interesting thing to explore.

> There is no interaction with the contents of the .git/objects
> directory and I'm not sure how that would work.

We'd watch .git/objects and .git/objects/{aa..ff}, then when about to
check for a loose object we'd avoid hitting the FS.

I don't know how useful that is post-61c7711cfea (sha1-file: use loose
object cache for quick existence check, 2018-11-12), but e.g. on NFS
this sort of thing still mattered. I had a "bigger hammer" approach with
[1] that ran (and still does, I believe) on a big corporate
installation.

More generally, if you strace .git access during repo operations you'll
find we're doing all sorts of existence checks etc. all the time. Loose
objects, refs, seeing what packs there are (better with the MIDX, but do
we still fall back?) etc. If we had up-to-date inotify/fsmonitor info we
could ask the daemon about it.

1. https://lore.kernel.org/git/20181028225023.26427-5-avarab@gmail.com/

Re: [PATCH] fsmonitor: only enable it in non-bare repositories

[Jeff Hostetler]

On 5/3/21 1:56 PM, Ævar Arnfjörð Bjarmason wrote:
> On Mon, May 03 2021, Jeff Hostetler wrote:
> 
>> On 5/3/21 9:58 AM, Ævar Arnfjörð Bjarmason wrote:
>>> On Thu, Apr 29 2021, Johannes Schindelin via GitGitGadget wrote:
>>>
>>>> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>>>>
>>>> The entire point of the FSMonitor is to monitor the worktree changes in
>>>> a more efficient manner than `lstat()`ing all worktree files every time
>>>> we refresh the index.
>>>>
>>>> But if there is no worktree, FSMonitor has nothing to monitor.
>>>>
>>>> So let's ignore if an FSMonitor is configured (e.g. in `~/.gitconfig`)
>>>> and we're running in a repository without worktree.
>>>>
>>>> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
>>>> ---
>> ...
>>>>
>>>> base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
>>> This is surely a correct fix for now, but wouldn't it in the future
>>> also
>>> be useful to run it in bare repositories e.g. to be able cache lookups
>>> for non-existing loose objects?
>>>
>>
>> No, the FSMonitor feature only expects data for paths within the
>> working directory.  (And is independent of whether the FS change
>> data is provided by my fsmonitor--daemon or provided by a hook-based
>> provider, such as Watchman.)  The FSMonitor feature uses that data to
>> shortcut scans of the working directory.
> 
> Indeed, hence "in the future". I'm not suggesting that it'll do anything
> useful by watching anything in the .git directory now, but that it might
> be an interesting thing to explore.
> 
>> There is no interaction with the contents of the .git/objects
>> directory and I'm not sure how that would work.
> 
> We'd watch .git/objects and .git/objects/{aa..ff}, then when about to
> check for a loose object we'd avoid hitting the FS.
> 
> I don't know how useful that is post-61c7711cfea (sha1-file: use loose
> object cache for quick existence check, 2018-11-12), but e.g. on NFS
> this sort of thing still mattered. I had a "bigger hammer" approach with
> [1] that ran (and still does, I believe) on a big corporate
> installation.
> 
> More generally, if you strace .git access during repo operations you'll
> find we're doing all sorts of existence checks etc. all the time. Loose
> objects, refs, seeing what packs there are (better with the MIDX, but do
> we still fall back?) etc. If we had up-to-date inotify/fsmonitor info we
> could ask the daemon about it.
> 
> 1. https://lore.kernel.org/git/20181028225023.26427-5-avarab@gmail.com/
> 

Interesting.

That's certainly something to look into later.  I know there is
code in the object lookup code to rescan/reload the packfiles or
loose objects (under the assumption that another process just created
a new packfile (and after our process loaded the packed-git list)).
Such a daemon might be helpful to improve something like that.

But I can't think about any of that right now.  I'd like to finish the
current fsmonitor--daemon patch series and let it settle down
before starting to think about an orthogonal use case such as this.

Thanks
Jeff