core.fsmonitor always perform rescans

core.fsmonitor always perform rescans

[Tatsuyuki Ishi]

Hello,

I'm facing issue with core.fsmonitor.

I'm currently using the provided watchman hook, but it doesn't seem to
record the fact that it has queried the fsmonitor backend, and as a
result the timestamp passed to the hook doesn't seem to change.

As it always pass a timestamp before watchman has crawled the
directories, watchman will always return all files inside the
directory. This happens everytime I run a git command, resulting in
slowness.

Is the timestamp not being updated an intended behavior, or is this a bug?

Tatsuyuki Ishi

Re: core.fsmonitor always perform rescans

[Johannes Schindelin]

Hi,

On Mon, 26 Mar 2018, Tatsuyuki Ishi wrote:

> I'm facing issue with core.fsmonitor.
> 
> I'm currently using the provided watchman hook, but it doesn't seem to
> record the fact that it has queried the fsmonitor backend, and as a
> result the timestamp passed to the hook doesn't seem to change.
> 
> As it always pass a timestamp before watchman has crawled the
> directories, watchman will always return all files inside the
> directory. This happens everytime I run a git command, resulting in
> slowness.
> 
> Is the timestamp not being updated an intended behavior, or is this a bug?

There are some bug fixes pending, and IIRC Alex was even working on some
more patches that are in an intermediate stage. Alex, could you share them
with us (even in their unfinished state, maybe others can help)?

Ciao,
Johannes

Re: core.fsmonitor always perform rescans

[Ben Peart]

On 3/26/2018 12:41 AM, Tatsuyuki Ishi wrote:
> Hello,
> 
> I'm facing issue with core.fsmonitor.
> 
> I'm currently using the provided watchman hook, but it doesn't seem to
> record the fact that it has queried the fsmonitor backend, and as a
> result the timestamp passed to the hook doesn't seem to change.
> 
> As it always pass a timestamp before watchman has crawled the
> directories, watchman will always return all files inside the
> directory. This happens everytime I run a git command, resulting in
> slowness.
> 
> Is the timestamp not being updated an intended behavior, or is this a bug?
> 

As a performance optimization, the fsmonitor code doesn't flag the index 
as dirty and force it to be written out with every command.  Can you try 
performing a git operation (add, rm, commit, etc) that will write out an 
updated index and see if that fixes the issue you're seeing?

I'm considering adding a special case to force the index to be written 
out the first time fsmonitor is invoked and am interested to know if 
this would have avoided the issue you are seeing.

Thanks!

> Tatsuyuki Ishi
> 

Re: core.fsmonitor always perform rescans

[Tatsuyuki Ishi]

2018-04-10 23:34 GMT+09:00 Ben Peart <peartben@gmail.com>:
> As a performance optimization, the fsmonitor code doesn't flag the index as
> dirty and force it to be written out with every command.  Can you try
> performing a git operation (add, rm, commit, etc) that will write out an
> updated index and see if that fixes the issue you're seeing?

Yeah, that resolves the issue. Though the repo I'm working on uses
submodules, so doing this in each of them isn't a easy work.

> I'm considering adding a special case to force the index to be written out
> the first time fsmonitor is invoked and am interested to know if this would
> have avoided the issue you are seeing.

Yes please. And maybe we should also flush the index when the script
returns '/',
which means all files?