Re: [PATCH v2 00/27] Builtin FSMonitor Part 3

[Jeff Hostetler <git@jeffhostetler.com>]

On 3/22/22 10:25 AM, rsbecker@nexbridge.com wrote:
> On March 22, 2022 10:11 AM, Jeff Hostetler wrote:
>> On 3/21/22 7:18 PM, rsbecker@nexbridge.com wrote:
>>> On March 21, 2022 6:06 PM, Jeff Hostetler wrote:
>>>> On 3/13/22 6:42 AM, Torsten Bögershausen wrote:
>>>>> Hej Jeff,
>>>>>
>> [...]
>>>>
>>>> It looks like the ...Cloned bit was added to the SDK in 10.13 [1].
>>>> All the other bits were defined sometime between 10.5 and 10.10.
>>>>
>>>> I'll add something in V7 to guard that bit.  I think 10.10 is old
>>>> enough that we don't need to special case those bits too.
>>>
>>> I realize it is a bit late in the game, but would you consider a pre-hook and post-
>> hook that automatically run with fsmonitor kicks off/terminates. I am thinking
>> about use cases where this is integrated into more complex processes and it
>> would be nice to have notifications of what fsmonitor is doing and when.
>>>
>>> Thanks,
>>> Randall
>>>
>>
>> I hadn't really considered having a pre/post hook for the daemon.
>> I'm not opposed to it; I just hadn't thought about it.
>>
>> By this I assume you mean something inside the fsmonitor--daemon process that
>> invokes the hooks when it is starting/stopping.
>> As opposed to something in a client command (like status) before it implicitly
>> started a daemon process.  The latter method would not give you post-hook
>> events because the daemon usually outlives the client command.
>>
>> Perhaps you could elaborate on what you would use these hooks for or how they
>> would be helpful.  It would be easy to add pre/post hooks in the main thread of
>> the daemon.  However, I worry about the prehook slowing the startup of the
>> daemon -- since the client status command might be waiting for it to become
>> ready.  I also have a "health" thread in part3 that would be a candidate for
>> pre/post and any other periodic hooks that might be useful.
>> But again, before I suggest a design for this, it would be good to know what kind of
>> things you would want to do with them.
> 
> Some examples of what I have in mind. There are more, but this covers what I have in mind urgently:
> 
> 1. Setting up a lock file (semaphore) just before fsmonitor runs that will cause any scripts that might change the state of the repository on the fly to suspend until fsmonitor is done.

The builtin fsmonitor--daemon is a long-running process.  It is
either explicitly started by the user or implicitly started by
clients commands, like "git status", lazily after it is enabled
in the config.  It runs until explicitly stopped (or the workdir
is deleted).

It is designed to be running and watch the filesystem for changes.

Later client commands can ask it for what has changed on disk
since the previous request (checkpoint).  And the only way to
capture that info is to watch the file system as things happen.
(Unless we have a really deep journal, but that often requires
admin access, so we don't use that.)

The fsmonitor daemon is unlike other subordinate commands in Git.
For example, "git fetch" might synchronously invoke "git index-pack"
and communicate over the child's stdin/stdout.  And that child is
bound to a single parent process.

When the daemon starts, it disassociates from the console and
opens a socket or named pipe and listens for requests (REST-like) 
commands.  It is designed to respond to multiple clients concurrently
and over a long time period -- like a daemon or service process.

> 2. Ensuring that in-flight scripts that do stuff are finished or not leaving the repo in a transitional state before fsmonitor runs - holding fsmonitor until the pre-hook finishes.
> 3. Notifying syslog or some other paging system if something has gone horribly wrong - as in fsmonitor found something bad in the index.

fsmonitor doesn't read the index.  it's only watching and summarizing
file system events.  And can enumerate the list of changed paths between
two checkpoints in response to a client request.


> 4. Clearing any semaphores created earlier (example 1).

The daemon isn't designed to ever "be done" and hence there are no
on-disk lock files/semaphores.  There might be some opportunities
for startup/shutdown to do some cleanup (temp files and the like),
but I think it is premature to talk about that right now.

Hope this helps,
Jeff


> 
> Regards,
> Randall
>