Windows: core.useBuiltinFSMonitor without core.untrackedcache - performance hazard?

[Tao Klerks <tao@klerks.biz>]

Hi folks,

With the new "core.useBuiltinFSMonitor" support in the Windows
installer, I think this subject is worth calling out explicitly (and
my apologies if I missed prior discussion):

TL;DR:
 - I believe "core.untrackedcache" should be enabled by default in
Windows (and it does not appear to be)
 - If a user enables "core.useBuiltinFSMonitor" (eg in the installer)
in the hopes of getting snappy "git status" on a repo with a large
deep working tree, they will be *unnecessarily* disappointed if
"core.untrackedcache" is not enabled
 - There is also a lingering "problem" with "git status -uall", with
both "core.useBuiltinFSMonitor" and "core.fsmonitor", but that seems
far less trivial to address

Detail:

I just started testing the new "core.useBuiltinFSMonitor" option in
the new installer, and it's amazing, thanks Ben, Alex, Johannes and
Kevin!

However, when I first enabled it, I was getting slightly *worse* git
status times than without it... and those worse git status times were
accompanied by a message along the lines of:
---
It took 5.88 seconds to enumerate untracked files. 'status -uno' may
speed it up, but you have to be careful not to forget to add new files
yourself (see 'git help status').
---

For context, this is in a repo with 200,000 or so files, within 40,000
folders (avg path depth 4 I think?), with a reasonably-intricate set
of .gitignore patterns. Obviously that's not "your average user", but
I would imagine it matches "the target audience for
'core.useBuiltinFSMonitor'" pretty well.

After a little head-scratching, I recalled an exchange with Johannes
from last year:
https://lore.kernel.org/git/CAPMMpohJicVeCaKsPvommYbGEH-D1V02TTMaiVTV8ux+9z9vkQ@mail.gmail.com/

I never did understand the relevant code paths in much detail, but the
practical conclusions were:
 - Without "core.untrackedcache" enabled, git ends up iterating
through the entire path structure of the working tree *even if
"core.fsmonitor" (and now "core.useBuiltinFSMonitor") is enabled*,
looking for untracked files to report
 - Even with "core.untrackedcache" enabled, if "core.fsmonitor" (and
now "core.useBuiltinFSMonitor") is enabled, git iterates through the
entire path structure of the working tree *single-threaded* when the
"--untracked-files" mode is set to "all" (by config or command-line)

Now, I imagine that addressing/improving these behaviors is very
non-trivial, but the impact could be reasonably limited if:
 - core.untrackedcache were defaulted to "true", at least under
Windows, at least when the installer is asked to set
core.useBuiltinFSMonitor
 - The "It took N.NN seconds to enumerate untracked files" message
were to include a hint about core.untrackedcache, at least when the
"--untracked-files" mode is set to "normal".

Final note: I personally would love to see "core.useBuiltinFSMonitor
actually makes things slower, when --untracked-files=all is specified"
behavior be addressed, because common windows git integrations or
front-ends like Git Extensions or IntelliJ IDEA commonly use those
options, and therefore "suffer" a performance degradation on at least
some operations when core.useBuiltinFSMonitor is enabled.

I don't know whether this is the right place to report Windows-centric
concerns, if not, my apologies.

Thanks,
Tao