git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: "Đoàn Trần Công Danh" <congdanhqx@gmail.com>
To: Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Cc: git@vger.kernel.org, jrnieder@gmail.com,
	jonathantanmy@google.com, sluongng@gmail.com,
	Derrick Stolee <derrickstolee@github.com>,
	Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH 2/7] maintenance: add --schedule option and config
Date: Tue, 8 Sep 2020 20:07:49 +0700	[thread overview]
Message-ID: <20200908130738.GB25593@danh.dev> (raw)
In-Reply-To: <1783e80b8d3b8361d1d62947a49ba584685dacc4.1599234126.git.gitgitgadget@gmail.com>

On 2020-09-04 15:42:01+0000, Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com> wrote:
> From: Derrick Stolee <dstolee@microsoft.com>
> 
> A user may want to run certain maintenance tasks based on frequency, not
> conditions given in the repository. For example, the user may want to

Hm, sorry but I couldn't decipher "not conditions" here. :|

> perform a 'prefetch' task every hour, or 'gc' task every day. To assist,

I think it's better to say: "To assist those users", at least it's
easier to read for non-native English like me.

> update the 'git maintenance run' command to include a
> '--schedule=<frequency>' option. The allowed frequencies are 'hourly',

So, we have "--schedule=" here, ...

> 'daily', and 'weekly'. These values are also allowed in a new config
> value 'maintenance.<task>.schedule'.
> 
> The 'git maintenance run --schedule=<frequency>' checks the '*.schedule'

and here, ...

> config value for each enabled task to see if the configured frequency is
> at least as frequent as the frequency from the '--schedule' argument. We
> use the following order, for full clarity:
> 
> 	'hourly' > 'daily' > 'weekly'
> 
> Use new 'enum schedule_priority' to track these values numerically.
> 
> The following cron table would run the scheduled tasks with the correct
> frequencies:
> 
>   0 1-23 * * *    git -C <repo> maintenance run --scheduled=hourly
>   0 0    * * 1-6  git -C <repo> maintenance run --scheduled=daily
>   0 0    * * 0    git -C <repo> maintenance run --scheduled=weekly

but it's spelt with "--scheduled=", here and below, mispell, I guess.

Reading the patch, it looks like "--scheduled=" is mispelt.

> This cron schedule will run --scheduled=hourly every hour except at
> midnight. This avoids a concurrent run with the --scheduled=daily that
> runs at midnight every day except the first day of the week. This avoids
> a concurrent run with the --scheduled=weekly that runs at midnight on
> the first day of the week. Since --scheduled=daily also runs the
> 'hourly' tasks and --scheduled=weekly runs the 'hourly' and 'daily'
> tasks, we will still see all tasks run with the proper frequencies.
> 
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  Documentation/config/maintenance.txt |  5 +++
>  Documentation/git-maintenance.txt    | 13 +++++-
>  builtin/gc.c                         | 67 +++++++++++++++++++++++++---
>  t/t7900-maintenance.sh               | 40 +++++++++++++++++
>  4 files changed, 119 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/config/maintenance.txt b/Documentation/config/maintenance.txt
> index 06db758172..70585564fa 100644
> --- a/Documentation/config/maintenance.txt
> +++ b/Documentation/config/maintenance.txt
> @@ -10,6 +10,11 @@ maintenance.<task>.enabled::
>  	`--task` option exists. By default, only `maintenance.gc.enabled`
>  	is true.
>  
> +maintenance.<task>.schedule::
> +	This config option controls whether or not the given `<task>` runs
> +	during a `git maintenance run --schedule=<frequency>` command. The
> +	value must be one of "hourly", "daily", or "weekly".
> +
>  maintenance.commit-graph.auto::
>  	This integer config option controls how often the `commit-graph` task
>  	should be run as part of `git maintenance run --auto`. If zero, then
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> index b44efb05a3..3af5907b01 100644
> --- a/Documentation/git-maintenance.txt
> +++ b/Documentation/git-maintenance.txt
> @@ -107,7 +107,18 @@ OPTIONS
>  	only if certain thresholds are met. For example, the `gc` task
>  	runs when the number of loose objects exceeds the number stored
>  	in the `gc.auto` config setting, or when the number of pack-files
> -	exceeds the `gc.autoPackLimit` config setting.
> +	exceeds the `gc.autoPackLimit` config setting. Not compatible with
> +	the `--schedule` option.
> +
> +--schedule::
> +	When combined with the `run` subcommand, run maintenance tasks
> +	only if certain time conditions are met, as specified by the
> +	`maintenance.<task>.schedule` config value for each `<task>`.
> +	This config value specifies a number of seconds since the last
> +	time that task ran, according to the `maintenance.<task>.lastRun`
> +	config value. The tasks that are tested are those provided by
> +	the `--task=<task>` option(s) or those with
> +	`maintenance.<task>.enabled` set to true.
>  
>  --quiet::
>  	Do not report progress or other information over `stderr`.
> diff --git a/builtin/gc.c b/builtin/gc.c
> index f8459df04c..85a3370692 100644
> --- a/builtin/gc.c
> +++ b/builtin/gc.c
> @@ -704,14 +704,51 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
>  	return 0;
>  }
>  
> -static const char * const builtin_maintenance_run_usage[] = {
> -	N_("git maintenance run [--auto] [--[no-]quiet] [--task=<task>]"),
> +static const char *const builtin_maintenance_run_usage[] = {
> +	N_("git maintenance run [--auto] [--[no-]quiet] [--task=<task>] [--schedule]"),
>  	NULL
>  };
>  
> +enum schedule_priority {
> +	SCHEDULE_NONE = 0,
> +	SCHEDULE_WEEKLY = 1,
> +	SCHEDULE_DAILY = 2,
> +	SCHEDULE_HOURLY = 3,
> +};
> +
> +static enum schedule_priority parse_schedule(const char *value)
> +{
> +	if (!value)
> +		return SCHEDULE_NONE;
> +	if (!strcasecmp(value, "hourly"))
> +		return SCHEDULE_HOURLY;
> +	if (!strcasecmp(value, "daily"))
> +		return SCHEDULE_DAILY;
> +	if (!strcasecmp(value, "weekly"))
> +		return SCHEDULE_WEEKLY;
> +	return SCHEDULE_NONE;
> +}
> +
> +static int maintenance_opt_schedule(const struct option *opt, const char *arg,
> +				    int unset)
> +{
> +	enum schedule_priority *priority = opt->value;
> +
> +	if (unset)
> +		die(_("--no-schedule is not allowed"));
> +
> +	*priority = parse_schedule(arg);
> +
> +	if (!*priority)
> +		die(_("unrecognized --schedule argument '%s'"), arg);
> +
> +	return 0;
> +}
> +
>  struct maintenance_run_opts {
>  	int auto_flag;
>  	int quiet;
> +	enum schedule_priority schedule;
>  };
>  
>  /* Remember to update object flag allocation in object.h */
> @@ -1159,6 +1196,8 @@ struct maintenance_task {
>  	maintenance_auto_fn *auto_condition;
>  	unsigned enabled:1;
>  
> +	enum schedule_priority schedule;
> +
>  	/* -1 if not selected. */
>  	int selected_order;
>  };
> @@ -1250,8 +1289,10 @@ static int maintenance_run_tasks(struct maintenance_run_opts *opts)
>  			continue;
>  
>  		if (opts->auto_flag &&
> -		    (!tasks[i].auto_condition ||
> -		     !tasks[i].auto_condition()))
> +		    (!tasks[i].auto_condition || !tasks[i].auto_condition()))
> +			continue;

This line only add unnecessary noise to this patch.

-- 
Danh
> +
> +		if (opts->schedule && tasks[i].schedule < opts->schedule)
>  			continue;
>  
>  		trace2_region_enter("maintenance", tasks[i].name, r);
> @@ -1274,13 +1315,23 @@ static void initialize_task_config(void)
>  
>  	for (i = 0; i < TASK__COUNT; i++) {
>  		int config_value;
> +		char *config_str;
>  
> -		strbuf_setlen(&config_name, 0);
> +		strbuf_reset(&config_name);
>  		strbuf_addf(&config_name, "maintenance.%s.enabled",
>  			    tasks[i].name);
>  
>  		if (!git_config_get_bool(config_name.buf, &config_value))
>  			tasks[i].enabled = config_value;
> +
> +		strbuf_reset(&config_name);
> +		strbuf_addf(&config_name, "maintenance.%s.schedule",
> +			    tasks[i].name);
> +
> +		if (!git_config_get_string(config_name.buf, &config_str)) {
> +			tasks[i].schedule = parse_schedule(config_str);
> +			free(config_str);
> +		}
>  	}
>  
>  	strbuf_release(&config_name);
> @@ -1324,6 +1375,9 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
>  	struct option builtin_maintenance_run_options[] = {
>  		OPT_BOOL(0, "auto", &opts.auto_flag,
>  			 N_("run tasks based on the state of the repository")),
> +		OPT_CALLBACK(0, "schedule", &opts.schedule, N_("frequency"),
> +			     N_("run tasks based on frequency"),
> +			     maintenance_opt_schedule),
>  		OPT_BOOL(0, "quiet", &opts.quiet,
>  			 N_("do not report progress or other information over stderr")),
>  		OPT_CALLBACK_F(0, "task", NULL, N_("task"),
> @@ -1344,6 +1398,9 @@ static int maintenance_run(int argc, const char **argv, const char *prefix)
>  			     builtin_maintenance_run_usage,
>  			     PARSE_OPT_STOP_AT_NON_OPTION);
>  
> +	if (opts.auto_flag && opts.schedule)
> +		die(_("use at most one of --auto and --schedule=<frequency>"));
> +
>  	if (argc != 0)
>  		usage_with_options(builtin_maintenance_run_usage,
>  				   builtin_maintenance_run_options);
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> index e0ba19e1ff..328bbaa830 100755
> --- a/t/t7900-maintenance.sh
> +++ b/t/t7900-maintenance.sh
> @@ -264,4 +264,44 @@ test_expect_success 'maintenance.incremental-repack.auto' '
>  	done
>  '
>  
> +test_expect_success '--auto and --schedule incompatible' '
> +	test_must_fail git maintenance run --auto --schedule=daily 2>err &&
> +	test_i18ngrep "at most one" err
> +'
> +
> +test_expect_success 'invalid --schedule value' '
> +	test_must_fail git maintenance run --schedule=annually 2>err &&
> +	test_i18ngrep "unrecognized --schedule" err
> +'
> +
> +test_expect_success '--schedule inheritance weekly -> daily -> hourly' '
> +	git config maintenance.loose-objects.enabled true &&
> +	git config maintenance.loose-objects.schedule hourly &&
> +	git config maintenance.commit-graph.enabled true &&
> +	git config maintenance.commit-graph.schedule daily &&
> +	git config maintenance.incremental-repack.enabled true &&
> +	git config maintenance.incremental-repack.schedule weekly &&
> +
> +	GIT_TRACE2_EVENT="$(pwd)/hourly.txt" \
> +		git maintenance run --schedule=hourly 2>/dev/null &&
> +	test_subcommand git prune-packed --quiet <hourly.txt &&
> +	test_subcommand ! git commit-graph write --split --reachable \
> +		--no-progress <hourly.txt &&
> +	test_subcommand ! git multi-pack-index write --no-progress <hourly.txt &&
> +
> +	GIT_TRACE2_EVENT="$(pwd)/daily.txt" \
> +		git maintenance run --schedule=daily 2>/dev/null &&
> +	test_subcommand git prune-packed --quiet <daily.txt &&
> +	test_subcommand git commit-graph write --split --reachable \
> +		--no-progress <daily.txt &&
> +	test_subcommand ! git multi-pack-index write --no-progress <daily.txt &&
> +
> +	GIT_TRACE2_EVENT="$(pwd)/weekly.txt" \
> +		git maintenance run --schedule=weekly 2>/dev/null &&
> +	test_subcommand git prune-packed --quiet <weekly.txt &&
> +	test_subcommand git commit-graph write --split --reachable \
> +		--no-progress <weekly.txt &&
> +	test_subcommand git multi-pack-index write --no-progress <weekly.txt
> +'
> +
>  test_done
> -- 
> gitgitgadget
> 

-- 
Danh

  reply	other threads:[~2020-09-08 19:39 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-04 15:41 [PATCH 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget
2020-09-04 15:42 ` [PATCH 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-09-04 15:42 ` [PATCH 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget
2020-09-08 13:07   ` Đoàn Trần Công Danh [this message]
2020-09-09 12:14     ` Derrick Stolee
2020-09-04 15:42 ` [PATCH 3/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-09-04 15:42 ` [PATCH 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-09-04 15:42 ` [PATCH 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-09-08  6:29   ` SZEDER Gábor
2020-09-08 12:43     ` Derrick Stolee
2020-09-08 19:31     ` Junio C Hamano
2020-09-04 15:42 ` [PATCH 6/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget
2020-09-04 15:42 ` [PATCH 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget
2020-09-11 17:49 ` [PATCH v2 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget
2020-09-11 17:49   ` [PATCH v2 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-09-11 17:49   ` [PATCH v2 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget
2020-09-11 17:49   ` [PATCH v2 3/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-09-11 17:49   ` [PATCH v2 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-09-17 14:05     ` Đoàn Trần Công Danh
2020-09-11 17:49   ` [PATCH v2 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-09-11 17:49   ` [PATCH v2 6/7] maintenance: recommended schedule in register/start Derrick Stolee via GitGitGadget
2020-09-29 19:48     ` Martin Ågren
2020-09-30 20:11       ` Derrick Stolee
2020-10-01 20:38         ` Derrick Stolee
2020-10-02  0:38           ` Đoàn Trần Công Danh
2020-10-02  1:55             ` Derrick Stolee
2020-10-05 13:16               ` Đoàn Trần Công Danh
2020-10-05 18:17                 ` Derrick Stolee
2020-09-11 17:49   ` [PATCH v2 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget
2020-10-05 12:57   ` [PATCH v3 0/7] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 1/7] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 2/7] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 3/7] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 4/7] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 5/7] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-10-05 12:57     ` [PATCH v3 6/7] maintenance: use default schedule if not configured Derrick Stolee via GitGitGadget
2020-10-05 19:57       ` Martin Ågren
2020-10-08 13:32         ` Derrick Stolee
2020-10-05 12:57     ` [PATCH v3 7/7] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget
2020-10-15 17:21     ` [PATCH v4 0/8] Maintenance III: Background maintenance Derrick Stolee via GitGitGadget
2020-10-15 17:21       ` [PATCH v4 1/8] maintenance: optionally skip --auto process Derrick Stolee via GitGitGadget
2020-10-15 17:21       ` [PATCH v4 2/8] maintenance: add --schedule option and config Derrick Stolee via GitGitGadget
2021-02-09 14:06         ` Ævar Arnfjörð Bjarmason
2021-02-09 16:54           ` Derrick Stolee
2021-05-10 12:16             ` Ævar Arnfjörð Bjarmason
2021-05-10 18:42               ` Junio C Hamano
2020-10-15 17:21       ` [PATCH v4 3/8] for-each-repo: run subcommands on configured repos Derrick Stolee via GitGitGadget
2021-05-03 16:10         ` Andrzej Hunt
2021-05-03 17:01           ` Eric Sunshine
2021-05-03 19:26             ` Eric Sunshine
2021-05-03 19:43           ` Derrick Stolee
2020-10-15 17:22       ` [PATCH v4 4/8] maintenance: add [un]register subcommands Derrick Stolee via GitGitGadget
2020-10-15 17:22       ` [PATCH v4 5/8] maintenance: add start/stop subcommands Derrick Stolee via GitGitGadget
2020-12-09 18:51         ` Josh Steadmon
2020-12-09 19:16           ` Josh Steadmon
2020-12-09 21:59             ` Derrick Stolee
2020-12-10  0:13             ` Junio C Hamano
2020-12-10  1:52               ` Derrick Stolee
2020-12-10  6:54                 ` Junio C Hamano
2020-10-15 17:22       ` [PATCH v4 6/8] maintenance: create maintenance.strategy config Derrick Stolee via GitGitGadget
2020-10-15 17:22       ` [PATCH v4 7/8] maintenance: use 'incremental' strategy by default Derrick Stolee via GitGitGadget
2020-10-15 17:22       ` [PATCH v4 8/8] maintenance: add troubleshooting guide to docs Derrick Stolee via GitGitGadget

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200908130738.GB25593@danh.dev \
    --to=congdanhqx@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=dstolee@microsoft.com \
    --cc=git@vger.kernel.org \
    --cc=gitgitgadget@gmail.com \
    --cc=jonathantanmy@google.com \
    --cc=jrnieder@gmail.com \
    --cc=sluongng@gmail.com \
    --subject='Re: [PATCH 2/7] maintenance: add --schedule option and config' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.version-control.git
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.version-control.git
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git