From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS53758 23.128.96.0/24 X-Spam-Status: No, score=-4.0 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_LOW, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by dcvr.yhbt.net (Postfix) with ESMTP id CC7191F670 for ; Thu, 21 Oct 2021 14:25:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231679AbhJUO16 (ORCPT ); Thu, 21 Oct 2021 10:27:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231207AbhJUO1l (ORCPT ); Thu, 21 Oct 2021 10:27:41 -0400 Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBE91C061229 for ; Thu, 21 Oct 2021 07:25:18 -0700 (PDT) Received: by mail-wr1-x436.google.com with SMTP id v17so309037wrv.9 for ; Thu, 21 Oct 2021 07:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=3+nodDMuYSIHacgPfA8/CyFz09f1Ewbhcn2EARycBS4=; b=bcDHZ+KNQw5BMfB3SmEBFP4qVjoP3tVA4vdB3d9T8ZYFAR+1nSL4lzagV3wL4hw+VH qGMPTuAMjVB0GeIYGkkSiXJPbhC/XXR9jAx9fDcT+GuTErC9F2ptNpWqOzzDxkEcdl9j sNDjtuGirijigaIwbdFbuAY3KZTRBpDGrqlPeUEMNj1f1wdJRnw6SMhiutQDqU5G4f4j RjvVicUV/gCZc7jbdjMC78773kneQA6yT/yGAFKBXpV1wukrZNYOmI6S4BuG+609cZL1 tKNkfhJ3MMU3A6E7ZJsKSR8Tejb8UjjZw1ZfKXgcMWNmk6Bmwhy0nD6ISAk055u9YDMg nA1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=3+nodDMuYSIHacgPfA8/CyFz09f1Ewbhcn2EARycBS4=; b=iajGJDdr3AjqvrWTFgsUTTsFlNpm1iTBNJWpA/F0PhtQCnLlpFb9xg7SS+YUsma/// tAJcGBDL6r1pXj3wS92E8fOtv6qalSpLm8ONugDdCmps2mTkmhasJaSfyPSzy1L1d1im EPTrv39BOSf1vUpy2V/Pko4tFR74KMwjHuSx9HuEKyYYzDngDY3FD/NUehma5wUZ+LyH 7PQdfd6F3kz84BpQKe2P8LdPFkFRW2SM3LQdoKlrImB1Y6un7Z8vUQfuIj7gpRpsVYg4 UwWSQaj2YC+zjvzR0QLpPzEgkKju+0p3KCYYRfU1becL1rIXm3QKuAn9UaBAVyT2VDAu 57BA== X-Gm-Message-State: AOAM533MtK5uL/DOf+1hYFPJT/b6bz+Wml4Rv8Mdlnzo+QU2qVTcVO/j M+yauRAdMdlNuPWv1MY2h/VwKZWD+dc= X-Google-Smtp-Source: ABdhPJxkjIOgh4ZPDKI20hNTS8/+GwILd6ELVdcHHx37dT8YoWbAYRyIpkebqW9G/8+0+eorzmhxug== X-Received: by 2002:adf:b31c:: with SMTP id j28mr7731120wrd.0.1634826317394; Thu, 21 Oct 2021 07:25:17 -0700 (PDT) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id p12sm6001900wrr.67.2021.10.21.07.25.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Oct 2021 07:25:17 -0700 (PDT) Message-Id: In-Reply-To: References: From: "Jeff Hostetler via GitGitGadget" Date: Thu, 21 Oct 2021 14:24:50 +0000 Subject: [PATCH v4 10/29] fsmonitor--daemon: implement 'run' command Fcc: Sent Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 To: git@vger.kernel.org Cc: Jeff Hostetler , Jeff Hostetler , Jeff Hostetler Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Jeff Hostetler Implement `run` command to try to begin listening for file system events. This version defines the thread structure with a single fsmonitor_fs_listen thread to watch for file system events and a simple IPC thread pool to watch for connection from Git clients over a well-known named pipe or Unix domain socket. This commit does not actually do anything yet because the platform backends are still just stubs. Signed-off-by: Jeff Hostetler --- builtin/fsmonitor--daemon.c | 213 +++++++++++++++++++++++++++++++++++- fsmonitor--daemon.h | 34 ++++++ 2 files changed, 246 insertions(+), 1 deletion(-) create mode 100644 fsmonitor--daemon.h diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 5e3178b8bdd..b5ebd1eca33 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -3,16 +3,39 @@ #include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "compat/fsmonitor/fsm-listen.h" +#include "fsmonitor--daemon.h" #include "simple-ipc.h" #include "khash.h" static const char * const builtin_fsmonitor__daemon_usage[] = { + N_("git fsmonitor--daemon run []"), N_("git fsmonitor--daemon stop"), N_("git fsmonitor--daemon status"), NULL }; #ifdef HAVE_FSMONITOR_DAEMON_BACKEND +/* + * Global state loaded from config. + */ +#define FSMONITOR__IPC_THREADS "fsmonitor.ipcthreads" +static int fsmonitor__ipc_threads = 8; + +static int fsmonitor_config(const char *var, const char *value, void *cb) +{ + if (!strcmp(var, FSMONITOR__IPC_THREADS)) { + int i = git_config_int(var, value); + if (i < 1) + return error(_("value of '%s' out of range: %d"), + FSMONITOR__IPC_THREADS, i); + fsmonitor__ipc_threads = i; + return 0; + } + + return git_default_config(var, value, cb); +} + /* * Acting as a CLIENT. * @@ -57,15 +80,196 @@ static int do_as_client__status(void) } } +static ipc_server_application_cb handle_client; + +static int handle_client(void *data, + const char *command, size_t command_len, + ipc_server_reply_cb *reply, + struct ipc_server_reply_data *reply_data) +{ + /* struct fsmonitor_daemon_state *state = data; */ + int result; + + /* + * The Simple IPC API now supports {char*, len} arguments, but + * FSMonitor always uses proper null-terminated strings, so + * we can ignore the command_len argument. (Trust, but verify.) + */ + if (command_len != strlen(command)) + BUG("FSMonitor assumes text messages"); + + trace2_region_enter("fsmonitor", "handle_client", the_repository); + trace2_data_string("fsmonitor", the_repository, "request", command); + + result = 0; /* TODO Do something here. */ + + trace2_region_leave("fsmonitor", "handle_client", the_repository); + + return result; +} + +static void *fsm_listen__thread_proc(void *_state) +{ + struct fsmonitor_daemon_state *state = _state; + + trace2_thread_start("fsm-listen"); + + trace_printf_key(&trace_fsmonitor, "Watching: worktree '%s'", + state->path_worktree_watch.buf); + if (state->nr_paths_watching > 1) + trace_printf_key(&trace_fsmonitor, "Watching: gitdir '%s'", + state->path_gitdir_watch.buf); + + fsm_listen__loop(state); + + trace2_thread_exit(); + return NULL; +} + +static int fsmonitor_run_daemon_1(struct fsmonitor_daemon_state *state) +{ + struct ipc_server_opts ipc_opts = { + .nr_threads = fsmonitor__ipc_threads, + + /* + * We know that there are no other active threads yet, + * so we can let the IPC layer temporarily chdir() if + * it needs to when creating the server side of the + * Unix domain socket. + */ + .uds_disallow_chdir = 0 + }; + + /* + * Start the IPC thread pool before the we've started the file + * system event listener thread so that we have the IPC handle + * before we need it. + */ + if (ipc_server_run_async(&state->ipc_server_data, + fsmonitor_ipc__get_path(), &ipc_opts, + handle_client, state)) + return error_errno( + _("could not start IPC thread pool on '%s'"), + fsmonitor_ipc__get_path()); + + /* + * Start the fsmonitor listener thread to collect filesystem + * events. + */ + if (pthread_create(&state->listener_thread, NULL, + fsm_listen__thread_proc, state) < 0) { + ipc_server_stop_async(state->ipc_server_data); + ipc_server_await(state->ipc_server_data); + + return error(_("could not start fsmonitor listener thread")); + } + + /* + * The daemon is now fully functional in background threads. + * Wait for the IPC thread pool to shutdown (whether by client + * request or from filesystem activity). + */ + ipc_server_await(state->ipc_server_data); + + /* + * The fsmonitor listener thread may have received a shutdown + * event from the IPC thread pool, but it doesn't hurt to tell + * it again. And wait for it to shutdown. + */ + fsm_listen__stop_async(state); + pthread_join(state->listener_thread, NULL); + + return state->error_code; +} + +static int fsmonitor_run_daemon(void) +{ + struct fsmonitor_daemon_state state; + int err; + + memset(&state, 0, sizeof(state)); + + pthread_mutex_init(&state.main_lock, NULL); + state.error_code = 0; + state.current_token_data = NULL; + + /* Prepare to (recursively) watch the directory. */ + strbuf_init(&state.path_worktree_watch, 0); + strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree())); + state.nr_paths_watching = 1; + + /* + * We create and delete cookie files somewhere inside the .git + * directory to help us keep sync with the file system. If + * ".git" is not a directory, then is not inside the + * cone of , so set up a second watch to watch + * the so that we get events for the cookie files. + */ + strbuf_init(&state.path_gitdir_watch, 0); + strbuf_addbuf(&state.path_gitdir_watch, &state.path_worktree_watch); + strbuf_addstr(&state.path_gitdir_watch, "/.git"); + if (!is_directory(state.path_gitdir_watch.buf)) { + strbuf_reset(&state.path_gitdir_watch); + strbuf_addstr(&state.path_gitdir_watch, absolute_path(get_git_dir())); + state.nr_paths_watching = 2; + } + + /* + * Confirm that we can create platform-specific resources for the + * filesystem listener before we bother starting all the threads. + */ + if (fsm_listen__ctor(&state)) { + err = error(_("could not initialize listener thread")); + goto done; + } + + err = fsmonitor_run_daemon_1(&state); + +done: + pthread_mutex_destroy(&state.main_lock); + fsm_listen__dtor(&state); + + ipc_server_free(state.ipc_server_data); + + strbuf_release(&state.path_worktree_watch); + strbuf_release(&state.path_gitdir_watch); + + return err; +} + +static int try_to_run_foreground_daemon(void) +{ + /* + * Technically, we don't need to probe for an existing daemon + * process, since we could just call `fsmonitor_run_daemon()` + * and let it fail if the pipe/socket is busy. + * + * However, this method gives us a nicer error message for a + * common error case. + */ + if (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + die("fsmonitor--daemon is already running '%s'", + the_repository->worktree); + + printf(_("running fsmonitor-daemon in '%s'\n"), + the_repository->worktree); + fflush(stdout); + + return !!fsmonitor_run_daemon(); +} + int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix) { const char *subcmd; struct option options[] = { + OPT_INTEGER(0, "ipc-threads", + &fsmonitor__ipc_threads, + N_("use ipc worker threads")), OPT_END() }; - git_config(git_default_config, NULL); + git_config(fsmonitor_config, NULL); argc = parse_options(argc, argv, prefix, options, builtin_fsmonitor__daemon_usage, 0); @@ -73,6 +277,13 @@ int cmd_fsmonitor__daemon(int argc, const char **argv, const char *prefix) usage_with_options(builtin_fsmonitor__daemon_usage, options); subcmd = argv[0]; + if (fsmonitor__ipc_threads < 1) + die(_("invalid 'ipc-threads' value (%d)"), + fsmonitor__ipc_threads); + + if (!strcmp(subcmd, "run")) + return !!try_to_run_foreground_daemon(); + if (!strcmp(subcmd, "stop")) return !!do_as_client__send_stop(); diff --git a/fsmonitor--daemon.h b/fsmonitor--daemon.h new file mode 100644 index 00000000000..3009c1a83de --- /dev/null +++ b/fsmonitor--daemon.h @@ -0,0 +1,34 @@ +#ifndef FSMONITOR_DAEMON_H +#define FSMONITOR_DAEMON_H + +#ifdef HAVE_FSMONITOR_DAEMON_BACKEND + +#include "cache.h" +#include "dir.h" +#include "run-command.h" +#include "simple-ipc.h" +#include "thread-utils.h" + +struct fsmonitor_batch; +struct fsmonitor_token_data; + +struct fsmonitor_daemon_backend_data; /* opaque platform-specific data */ + +struct fsmonitor_daemon_state { + pthread_t listener_thread; + pthread_mutex_t main_lock; + + struct strbuf path_worktree_watch; + struct strbuf path_gitdir_watch; + int nr_paths_watching; + + struct fsmonitor_token_data *current_token_data; + + int error_code; + struct fsmonitor_daemon_backend_data *backend_data; + + struct ipc_server_data *ipc_server_data; +}; + +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ +#endif /* FSMONITOR_DAEMON_H */ -- gitgitgadget