git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Đoàn Trần Công Danh" <congdanhqx@gmail.com>
To: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, sandals@crustytoothpaste.net,
	steadmon@google.com, jrnieder@gmail.com, peff@peff.net,
	phillip.wood123@gmail.com, emilyshaffer@google.com,
	sluongng@gmail.com, jonathantanmy@google.com,
	Derrick Stolee <derrickstolee@github.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH 4/7] for-each-repo: run subcommands on configured repos
Date: Thu, 20 Aug 2020 22:00:48 +0700	[thread overview]
Message-ID: <20200820150048.GD31084@danh.dev> (raw)
In-Reply-To: <ccb667dc6fa4704171da643158ecd79ed7dd2cd3.1597857412.git.gitgitgadget@gmail.com>

On 2020-08-19 17:16:45+0000, Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote:
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> It can be helpful to store a list of repositories in global or system
> config and then iterate Git commands on that list. Create a new builtin
> that makes this process simple for experts. We will use this builtin to
> run scheduled maintenance on all configured repositories in a future
> change.

Nice, I like this new command.

However, I'm not sure if we could declare this command as plumbing or
porcelain command.

I guess this command is meant more for scripting purpose, hence, it
should be plumbing, thus we need to define clear protocol for this
command, e.g, where it will redirect other command output, error to,
where for-each-repo write its own output/error.

Also, I think it would be nice to declare this is experimental for now.
Like we declared git-switch and git-restore.

-- 
Danh

> 
> The test is very simple, but does highlight that the "--" argument is
> optional.
> 
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  .gitignore                          |  1 +
>  Documentation/git-for-each-repo.txt | 45 ++++++++++++++++++++++
>  Makefile                            |  1 +
>  builtin.h                           |  1 +
>  builtin/for-each-repo.c             | 58 +++++++++++++++++++++++++++++
>  git.c                               |  1 +
>  t/t0068-for-each-repo.sh            | 30 +++++++++++++++
>  7 files changed, 137 insertions(+)
>  create mode 100644 Documentation/git-for-each-repo.txt
>  create mode 100644 builtin/for-each-repo.c
>  create mode 100755 t/t0068-for-each-repo.sh
> 
> diff --git a/.gitignore b/.gitignore
> index a5808fa30d..5eb2a2be71 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -67,6 +67,7 @@
>  /git-filter-branch
>  /git-fmt-merge-msg
>  /git-for-each-ref
> +/git-for-each-repo
>  /git-format-patch
>  /git-fsck
>  /git-fsck-objects
> diff --git a/Documentation/git-for-each-repo.txt b/Documentation/git-for-each-repo.txt
> new file mode 100644
> index 0000000000..83b06db410
> --- /dev/null
> +++ b/Documentation/git-for-each-repo.txt
> @@ -0,0 +1,45 @@
> +git-for-each-repo(1)
> +====================
> +
> +NAME
> +----
> +git-for-each-repo - Run a Git command on a list of repositories
> +
> +
> +SYNOPSIS
> +--------
> +[verse]
> +'git for-each-repo' --config=<config> [--] <arguments>
> +
> +
> +DESCRIPTION
> +-----------
> +Run a Git commands on a list of repositories. The arguments after the
> +known options or `--` indicator are used as the arguments for the Git
> +command.
> +
> +For example, we could run maintenance on each of a list of repositories
> +stored in a `maintenance.repo` config variable using
> +
> +-------------
> +git for-each-repo --config=maintenance.repo maintenance run
> +-------------
> +
> +This will run `git -C <repo> maintenance run` for each value `<repo>`
> +in the multi-valued config variable `maintenance.repo`.
> +
> +
> +OPTIONS
> +-------
> +--config=<config>::
> +	Use the given config variable as a multi-valued list storing
> +	absolute path names. Iterate on that list of paths to run
> +	the given arguments.
> ++
> +These config values are loaded from system, global, and local Git config,
> +as available. If `git for-each-repo` is run in a directory that is not a
> +Git repository, then only the system and global config is used.
> +
> +GIT
> +---
> +Part of the linkgit:git[1] suite
> diff --git a/Makefile b/Makefile
> index 65f8cfb236..7c588ff036 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1071,6 +1071,7 @@ BUILTIN_OBJS += builtin/fetch-pack.o
>  BUILTIN_OBJS += builtin/fetch.o
>  BUILTIN_OBJS += builtin/fmt-merge-msg.o
>  BUILTIN_OBJS += builtin/for-each-ref.o
> +BUILTIN_OBJS += builtin/for-each-repo.o
>  BUILTIN_OBJS += builtin/fsck.o
>  BUILTIN_OBJS += builtin/gc.o
>  BUILTIN_OBJS += builtin/get-tar-commit-id.o
> diff --git a/builtin.h b/builtin.h
> index 17c1c0ce49..ff7c6e5aa9 100644
> --- a/builtin.h
> +++ b/builtin.h
> @@ -150,6 +150,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix);
>  int cmd_fetch_pack(int argc, const char **argv, const char *prefix);
>  int cmd_fmt_merge_msg(int argc, const char **argv, const char *prefix);
>  int cmd_for_each_ref(int argc, const char **argv, const char *prefix);
> +int cmd_for_each_repo(int argc, const char **argv, const char *prefix);
>  int cmd_format_patch(int argc, const char **argv, const char *prefix);
>  int cmd_fsck(int argc, const char **argv, const char *prefix);
>  int cmd_gc(int argc, const char **argv, const char *prefix);
> diff --git a/builtin/for-each-repo.c b/builtin/for-each-repo.c
> new file mode 100644
> index 0000000000..5bba623ff1
> --- /dev/null
> +++ b/builtin/for-each-repo.c
> @@ -0,0 +1,58 @@
> +#include "cache.h"
> +#include "config.h"
> +#include "builtin.h"
> +#include "parse-options.h"
> +#include "run-command.h"
> +#include "string-list.h"
> +
> +static const char * const for_each_repo_usage[] = {
> +	N_("git for-each-repo --config=<config> <command-args>"),
> +	NULL
> +};
> +
> +static int run_command_on_repo(const char *path,
> +			       void *cbdata)
> +{
> +	int i;
> +	struct child_process child = CHILD_PROCESS_INIT;
> +	struct strvec *args = (struct strvec *)cbdata;
> +
> +	child.git_cmd = 1;
> +	strvec_pushl(&child.args, "-C", path, NULL);
> +
> +	for (i = 0; i < args->nr; i++)
> +		strvec_push(&child.args, args->v[i]);
> +
> +	return run_command(&child);
> +}
> +
> +int cmd_for_each_repo(int argc, const char **argv, const char *prefix)
> +{
> +	static const char *config_key = NULL;
> +	int i, result = 0;
> +	const struct string_list *values;
> +	struct strvec args = STRVEC_INIT;
> +
> +	const struct option options[] = {
> +		OPT_STRING(0, "config", &config_key, N_("config"),
> +			   N_("config key storing a list of repository paths")),
> +		OPT_END()
> +	};
> +
> +	argc = parse_options(argc, argv, prefix, options, for_each_repo_usage,
> +			     PARSE_OPT_STOP_AT_NON_OPTION);
> +
> +	if (!config_key)
> +		die(_("missing --config=<config>"));
> +
> +	for (i = 0; i < argc; i++)
> +		strvec_push(&args, argv[i]);
> +
> +	values = repo_config_get_value_multi(the_repository,
> +					     config_key);
> +
> +	for (i = 0; !result && i < values->nr; i++)
> +		result = run_command_on_repo(values->items[i].string, &args);
> +
> +	return result;
> +}
> diff --git a/git.c b/git.c
> index 24f250d29a..1cab64b5d1 100644
> --- a/git.c
> +++ b/git.c
> @@ -511,6 +511,7 @@ static struct cmd_struct commands[] = {
>  	{ "fetch-pack", cmd_fetch_pack, RUN_SETUP | NO_PARSEOPT },
>  	{ "fmt-merge-msg", cmd_fmt_merge_msg, RUN_SETUP },
>  	{ "for-each-ref", cmd_for_each_ref, RUN_SETUP },
> +	{ "for-each-repo", cmd_for_each_repo, RUN_SETUP_GENTLY },
>  	{ "format-patch", cmd_format_patch, RUN_SETUP },
>  	{ "fsck", cmd_fsck, RUN_SETUP },
>  	{ "fsck-objects", cmd_fsck, RUN_SETUP },
> diff --git a/t/t0068-for-each-repo.sh b/t/t0068-for-each-repo.sh
> new file mode 100755
> index 0000000000..136b4ec839
> --- /dev/null
> +++ b/t/t0068-for-each-repo.sh
> @@ -0,0 +1,30 @@
> +#!/bin/sh
> +
> +test_description='git for-each-repo builtin'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'run based on configured value' '
> +	git init one &&
> +	git init two &&
> +	git init three &&
> +	git -C two commit --allow-empty -m "DID NOT RUN" &&
> +	git config run.key "$TRASH_DIRECTORY/one" &&
> +	git config --add run.key "$TRASH_DIRECTORY/three" &&
> +	git for-each-repo --config=run.key commit --allow-empty -m "ran" &&
> +	git -C one log -1 --pretty=format:%s >message &&
> +	grep ran message &&
> +	git -C two log -1 --pretty=format:%s >message &&
> +	! grep ran message &&
> +	git -C three log -1 --pretty=format:%s >message &&
> +	grep ran message &&
> +	git for-each-repo --config=run.key -- commit --allow-empty -m "ran again" &&
> +	git -C one log -1 --pretty=format:%s >message &&
> +	grep again message &&
> +	git -C two log -1 --pretty=format:%s >message &&
> +	! grep again message &&
> +	git -C three log -1 --pretty=format:%s >message &&
> +	grep again message
> +'
> +
> +test_done
> -- 
> gitgitgadget
> 

-- 
Danh

  reply	other threads:[~2020-08-20 15:01 UTC|newest]

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-19 17:16 [PATCH 0/7] [RFC] Maintenance III: background maintenance Derrick Stolee via GitGitGadget
2020-08-19 17:16 ` [PATCH 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-08-20  2:06   ` Đoàn Trần Công Danh
2020-08-20 12:12     ` Derrick Stolee
2020-08-19 17:16 ` [PATCH 3/7] maintenance: add --scheduled option and config Derrick Stolee via GitGitGadget
2020-08-20 14:51   ` Đoàn Trần Công Danh
2020-08-24 14:03     ` Derrick Stolee
2020-08-19 17:16 ` [PATCH 4/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-08-20 15:00   ` Đoàn Trần Công Danh [this message]
2020-08-19 17:16 ` [PATCH 5/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-08-19 17:16 ` [PATCH 6/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-08-19 17:16 ` [PATCH 7/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget
     [not found] ` <bdc27fa28ee70222ed3c7c9863746ace8ea835e4.1597857409.git.gitgitgadget@gmail.com>
2020-08-20 14:34   ` [PATCH 2/7] maintenance: store the "last run" time in config Đoàn Trần Công Danh
2020-08-25 18:39 ` [PATCH v2 0/7] [RFC] Maintenance III: background maintenance Derrick Stolee via GitGitGadget
2020-08-25 18:39   ` [PATCH v2 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-08-25 21:44     ` Junio C Hamano
2020-08-26 12:29       ` Derrick Stolee
2020-08-26 16:57         ` Junio C Hamano
2020-08-25 18:39   ` [PATCH v2 2/7] maintenance: store the "last run" time in config Derrick Stolee via GitGitGadget
2020-08-25 21:52     ` Junio C Hamano
2020-08-26 13:34       ` Derrick Stolee
2020-08-26 17:03         ` Junio C Hamano
2020-08-27 13:02           ` Derrick Stolee
2020-08-25 18:40   ` [PATCH v2 3/7] maintenance: add --scheduled option and config Derrick Stolee via GitGitGadget
2020-08-25 22:01     ` Junio C Hamano
2020-08-26 15:30       ` Derrick Stolee
2020-08-27 15:47         ` Derrick Stolee
2020-08-25 18:40   ` [PATCH v2 4/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-08-25 22:19     ` Junio C Hamano
2020-08-26 16:03       ` Derrick Stolee
2020-08-25 18:40   ` [PATCH v2 5/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-08-25 18:40   ` [PATCH v2 6/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-08-25 18:40   ` [PATCH v2 7/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget
2020-08-28 15:45   ` [PATCH v3 0/6] [RFC] Maintenance III: background maintenance Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 1/6] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 2/6] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 3/6] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 4/6] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 5/6] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-08-28 15:45     ` [PATCH v3 6/6] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget
2020-08-26 12:42 ` [PATCH 0/7] [RFC] Maintenance III: background maintenance Michal Suchánek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200820150048.GD31084@danh.dev \
    --to=congdanhqx@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=dstolee@microsoft.com \
    --cc=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=jonathantanmy@google.com \
    --cc=jrnieder@gmail.com \
    --cc=peff@peff.net \
    --cc=phillip.wood123@gmail.com \
    --cc=sandals@crustytoothpaste.net \
    --cc=sluongng@gmail.com \
    --cc=steadmon@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).