From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> To: git@vger.kernel.org Cc: jrnieder@gmail.com, jonathantanmy@google.com, sluongng@gmail.com, congdanhqx@gmail.com, Derrick Stolee <derrickstolee@github.com>, Derrick Stolee <dstolee@microsoft.com> Subject: [PATCH 3/7] for-each-repo: run subcommands on configured repos Date: Fri, 04 Sep 2020 15:42:02 +0000 Message-ID: <6082d939eb6076303e68deb1c61d9313d62788ab.1599234126.git.gitgitgadget@gmail.com> (raw) In-Reply-To: <pull.724.git.1599234126.gitgitgadget@gmail.com> From: Derrick Stolee <dstolee@microsoft.com> It can be helpful to store a list of repositories in global or system config and then iterate Git commands on that list. Create a new builtin that makes this process simple for experts. We will use this builtin to run scheduled maintenance on all configured repositories in a future change. The test is very simple, but does highlight that the "--" argument is optional. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> --- .gitignore | 1 + Documentation/git-for-each-repo.txt | 59 +++++++++++++++++++++++++++++ Makefile | 1 + builtin.h | 1 + builtin/for-each-repo.c | 58 ++++++++++++++++++++++++++++ command-list.txt | 1 + git.c | 1 + t/t0068-for-each-repo.sh | 30 +++++++++++++++ 8 files changed, 152 insertions(+) create mode 100644 Documentation/git-for-each-repo.txt create mode 100644 builtin/for-each-repo.c create mode 100755 t/t0068-for-each-repo.sh diff --git a/.gitignore b/.gitignore index a5808fa30d..5eb2a2be71 100644 --- a/.gitignore +++ b/.gitignore @@ -67,6 +67,7 @@ /git-filter-branch /git-fmt-merge-msg /git-for-each-ref +/git-for-each-repo /git-format-patch /git-fsck /git-fsck-objects diff --git a/Documentation/git-for-each-repo.txt b/Documentation/git-for-each-repo.txt new file mode 100644 index 0000000000..94bd19da26 --- /dev/null +++ b/Documentation/git-for-each-repo.txt @@ -0,0 +1,59 @@ +git-for-each-repo(1) +==================== + +NAME +---- +git-for-each-repo - Run a Git command on a list of repositories + + +SYNOPSIS +-------- +[verse] +'git for-each-repo' --config=<config> [--] <arguments> + + +DESCRIPTION +----------- +Run a Git command on a list of repositories. The arguments after the +known options or `--` indicator are used as the arguments for the Git +subprocess. + +THIS COMMAND IS EXPERIMENTAL. THE BEHAVIOR MAY CHANGE. + +For example, we could run maintenance on each of a list of repositories +stored in a `maintenance.repo` config variable using + +------------- +git for-each-repo --config=maintenance.repo maintenance run +------------- + +This will run `git -C <repo> maintenance run` for each value `<repo>` +in the multi-valued config variable `maintenance.repo`. + + +OPTIONS +------- +--config=<config>:: + Use the given config variable as a multi-valued list storing + absolute path names. Iterate on that list of paths to run + the given arguments. ++ +These config values are loaded from system, global, and local Git config, +as available. If `git for-each-repo` is run in a directory that is not a +Git repository, then only the system and global config is used. + + +SUBPROCESS BEHAVIOR +------------------- + +If any `git -C <repo> <arguments>` subprocess returns a non-zero exit code, +then the `git for-each-repo` process returns that exit code without running +more subprocesses. + +Each `git -C <repo> <arguments>` subprocess inherits the standard file +descriptors `stdin`, `stdout`, and `stderr`. + + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Makefile b/Makefile index 65f8cfb236..7c588ff036 100644 --- a/Makefile +++ b/Makefile @@ -1071,6 +1071,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o BUILTIN_OBJS += builtin/fetch.o BUILTIN_OBJS += builtin/fmt-merge-msg.o BUILTIN_OBJS += builtin/for-each-ref.o +BUILTIN_OBJS += builtin/for-each-repo.o BUILTIN_OBJS += builtin/fsck.o BUILTIN_OBJS += builtin/gc.o BUILTIN_OBJS += builtin/get-tar-commit-id.o diff --git a/builtin.h b/builtin.h index 17c1c0ce49..ff7c6e5aa9 100644 --- a/builtin.h +++ b/builtin.h @@ -150,6 +150,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix); int cmd_fetch_pack(int argc, const char **argv, const char *prefix); int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix); int cmd_for_each_ref(int argc, const char **argv, const char *prefix); +int cmd_for_each_repo(int argc, const char **argv, const char *prefix); int cmd_format_patch(int argc, const char **argv, const char *prefix); int cmd_fsck(int argc, const char **argv, const char *prefix); int cmd_gc(int argc, const char **argv, const char *prefix); diff --git a/builtin/for-each-repo.c b/builtin/for-each-repo.c new file mode 100644 index 0000000000..5bba623ff1 --- /dev/null +++ b/builtin/for-each-repo.c @@ -0,0 +1,58 @@ +#include "cache.h" +#include "config.h" +#include "builtin.h" +#include "parse-options.h" +#include "run-command.h" +#include "string-list.h" + +static const char * const for_each_repo_usage[] = { + N_("git for-each-repo --config=<config> <command-args>"), + NULL +}; + +static int run_command_on_repo(const char *path, + void *cbdata) +{ + int i; + struct child_process child = CHILD_PROCESS_INIT; + struct strvec *args = (struct strvec *)cbdata; + + child.git_cmd = 1; + strvec_pushl(&child.args, "-C", path, NULL); + + for (i = 0; i < args->nr; i++) + strvec_push(&child.args, args->v[i]); + + return run_command(&child); +} + +int cmd_for_each_repo(int argc, const char **argv, const char *prefix) +{ + static const char *config_key = NULL; + int i, result = 0; + const struct string_list *values; + struct strvec args = STRVEC_INIT; + + const struct option options[] = { + OPT_STRING(0, "config", &config_key, N_("config"), + N_("config key storing a list of repository paths")), + OPT_END() + }; + + argc = parse_options(argc, argv, prefix, options, for_each_repo_usage, + PARSE_OPT_STOP_AT_NON_OPTION); + + if (!config_key) + die(_("missing --config=<config>")); + + for (i = 0; i < argc; i++) + strvec_push(&args, argv[i]); + + values = repo_config_get_value_multi(the_repository, + config_key); + + for (i = 0; !result && i < values->nr; i++) + result = run_command_on_repo(values->items[i].string, &args); + + return result; +} diff --git a/command-list.txt b/command-list.txt index 0e3204e7d1..581499be82 100644 --- a/command-list.txt +++ b/command-list.txt @@ -94,6 +94,7 @@ git-fetch-pack synchingrepositories git-filter-branch ancillarymanipulators git-fmt-merge-msg purehelpers git-for-each-ref plumbinginterrogators +git-for-each-repo plumbinginterrogators git-format-patch mainporcelain git-fsck ancillaryinterrogators complete git-gc mainporcelain diff --git a/git.c b/git.c index 24f250d29a..1cab64b5d1 100644 --- a/git.c +++ b/git.c @@ -511,6 +511,7 @@ static struct cmd_struct commands[] = { { "fetch-pack", cmd_fetch_pack, RUN_SETUP | NO_PARSEOPT }, { "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP }, { "for-each-ref", cmd_for_each_ref, RUN_SETUP }, + { "for-each-repo", cmd_for_each_repo, RUN_SETUP_GENTLY }, { "format-patch", cmd_format_patch, RUN_SETUP }, { "fsck", cmd_fsck, RUN_SETUP }, { "fsck-objects", cmd_fsck, RUN_SETUP }, diff --git a/t/t0068-for-each-repo.sh b/t/t0068-for-each-repo.sh new file mode 100755 index 0000000000..136b4ec839 --- /dev/null +++ b/t/t0068-for-each-repo.sh @@ -0,0 +1,30 @@ +#!/bin/sh + +test_description='git for-each-repo builtin' + +. ./test-lib.sh + +test_expect_success 'run based on configured value' ' + git init one && + git init two && + git init three && + git -C two commit --allow-empty -m "DID NOT RUN" && + git config run.key "$TRASH_DIRECTORY/one" && + git config --add run.key "$TRASH_DIRECTORY/three" && + git for-each-repo --config=run.key commit --allow-empty -m "ran" && + git -C one log -1 --pretty=format:%s >message && + grep ran message && + git -C two log -1 --pretty=format:%s >message && + ! grep ran message && + git -C three log -1 --pretty=format:%s >message && + grep ran message && + git for-each-repo --config=run.key -- commit --allow-empty -m "ran again" && + git -C one log -1 --pretty=format:%s >message && + grep again message && + git -C two log -1 --pretty=format:%s >message && + ! grep again message && + git -C three log -1 --pretty=format:%s >message && + grep again message +' + +test_done -- gitgitgadget
next prev parent reply other threads:[~2020-09-04 15:42 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-04 15:41 [PATCH 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget 2020-09-04 15:42 ` [PATCH 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget 2020-09-04 15:42 ` [PATCH 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget 2020-09-08 13:07 ` Đoàn Trần Công Danh 2020-09-09 12:14 ` Derrick Stolee 2020-09-04 15:42 ` Derrick Stolee via GitGitGadget [this message] 2020-09-04 15:42 ` [PATCH 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget 2020-09-04 15:42 ` [PATCH 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget 2020-09-08 6:29 ` SZEDER Gábor 2020-09-08 12:43 ` Derrick Stolee 2020-09-08 19:31 ` Junio C Hamano 2020-09-04 15:42 ` [PATCH 6/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget 2020-09-04 15:42 ` [PATCH 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 3/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget 2020-09-17 14:05 ` Đoàn Trần Công Danh 2020-09-11 17:49 ` [PATCH v2 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget 2020-09-11 17:49 ` [PATCH v2 6/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget 2020-09-29 19:48 ` Martin Ågren 2020-09-30 20:11 ` Derrick Stolee 2020-10-01 20:38 ` Derrick Stolee 2020-10-02 0:38 ` Đoàn Trần Công Danh 2020-10-02 1:55 ` Derrick Stolee 2020-10-05 13:16 ` Đoàn Trần Công Danh 2020-10-05 18:17 ` Derrick Stolee 2020-09-11 17:49 ` [PATCH v2 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 3/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget 2020-10-05 12:57 ` [PATCH v3 6/7] maintenance: use default schedule if not configured Derrick Stolee via GitGitGadget 2020-10-05 19:57 ` Martin Ågren 2020-10-08 13:32 ` Derrick Stolee 2020-10-05 12:57 ` [PATCH v3 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget 2020-10-15 17:21 ` [PATCH v4 0/8] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget 2020-10-15 17:21 ` [PATCH v4 1/8] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget 2020-10-15 17:21 ` [PATCH v4 2/8] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget 2020-10-15 17:21 ` [PATCH v4 3/8] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget 2020-10-15 17:22 ` [PATCH v4 4/8] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget 2020-10-15 17:22 ` [PATCH v4 5/8] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget 2020-12-09 18:51 ` Josh Steadmon 2020-12-09 19:16 ` Josh Steadmon 2020-12-09 21:59 ` Derrick Stolee 2020-12-10 0:13 ` Junio C Hamano 2020-12-10 1:52 ` Derrick Stolee 2020-12-10 6:54 ` Junio C Hamano 2020-10-15 17:22 ` [PATCH v4 6/8] maintenance: create maintenance.strategy config Derrick Stolee via GitGitGadget 2020-10-15 17:22 ` [PATCH v4 7/8] maintenance: use 'incremental' strategy by default Derrick Stolee via GitGitGadget 2020-10-15 17:22 ` [PATCH v4 8/8] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=6082d939eb6076303e68deb1c61d9313d62788ab.1599234126.git.gitgitgadget@gmail.com \ --to=gitgitgadget@gmail.com \ --cc=congdanhqx@gmail.com \ --cc=derrickstolee@github.com \ --cc=dstolee@microsoft.com \ --cc=git@vger.kernel.org \ --cc=jonathantanmy@google.com \ --cc=jrnieder@gmail.com \ --cc=sluongng@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
git@vger.kernel.org list mirror (unofficial, one of many) This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 git git/ https://public-inbox.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/mirrors/git.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git