[PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 2,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 3 describes a plan to get around that.

Thanks, -Stolee

Derrick Stolee (3):
  maintenance: extract platform-specific scheduling
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 builtin/gc.c           | 428 +++++++++++++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh |  86 ++++++++-
 t/test-lib.sh          |   4 +
 3 files changed, 498 insertions(+), 20 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/776
-- 
gitgitgadget

[PATCH 1/3] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c | 38 +++++++++++++++++++++-----------------
 1 file changed, 21 insertions(+), 17 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a1..c1f7d9bdc2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int platform_update_schedule(int run_maintenance, int fd)
 {
 	int result = 0;
 	int in_old_region = 0;
@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)
 	FILE *cron_list, *cron_in;
 	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
-
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
 
 	crontab_name = getenv("GIT_TEST_CRONTAB");
 	if (!crontab_name)
@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)
 	strvec_split(&crontab_list.args, crontab_name);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
 	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 	}
 
 	/* Ignore exit code, as an empty crontab will return error. */
@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
 	strvec_split(&crontab_edit.args, crontab_name);
@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)
 	crontab_edit.git_cmd = 0;
 
 	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 	}
 
 	cron_in = fdopen(crontab_edit.in, "w");
@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
-	}
-	fclose(cron_list);
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static int update_background_schedule(int run_maintenance)
+{
+	int result;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	result = platform_update_schedule(run_maintenance, lk.tempfile->fd);
 
-cleanup:
 	rollback_lock_file(&lk);
 	return result;
 }
-- 
gitgitgadget

[PATCH 2/3] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2].

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_CRONTAB.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 209 +++++++++++++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh |  52 +++++++++-
 t/test-lib.sh          |   4 +
 3 files changed, 262 insertions(+), 3 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index c1f7d9bdc2..fa0ae63a80 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+#if defined(__APPLE__)
+
+static char *get_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *get_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_uid(void)
+{
+	struct strbuf output = STRBUF_INIT;
+	struct child_process id = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL);
+	if (capture_command(&id, &output, 0))
+		die(_("failed to discover user id"));
+
+	strbuf_trim_trailing_newline(&output);
+	return strbuf_detach(&output, NULL);
+}
+
+static int bootout(const char *filename)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	char *uid = get_uid();
+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
+	if (!launchctl)
+		launchctl = "/bin/launchctl";
+
+	strvec_split(&args, launchctl);
+	strvec_push(&args, "bootout");
+	strvec_pushf(&args, "gui/%s", uid);
+	strvec_push(&args, filename);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(uid);
+	return result;
+}
+
+static int bootstrap(const char *filename)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	char *uid = get_uid();
+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
+	if (!launchctl)
+		launchctl = "/bin/launchctl";
+
+	strvec_split(&args, launchctl);
+	strvec_push(&args, "bootstrap");
+	strvec_pushf(&args, "gui/%s", uid);
+	strvec_push(&args, filename);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(uid);
+	return result;
+}
+
+static int remove_plist(enum schedule_priority schedule)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+	int result = bootout(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int remove_plists(void)
+{
+	return remove_plist(SCHEDULE_HOURLY) ||
+		remove_plist(SCHEDULE_DAILY) ||
+		remove_plist(SCHEDULE_WEEKLY);
+}
+
+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = fopen(filename, "w");
+
+	if (!plist)
+		die(_("failed to open '%s'"), filename);
+
+	preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "<dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+
+	/* bootout might fail if not already running, so ignore */
+	bootout(filename);
+	if (bootstrap(filename))
+		die(_("failed to bootstrap service %s"), filename);
+
+	fclose(plist);
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int add_plists(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		schedule_plist(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_plists();
+	else
+		return remove_plists();
+}
+#else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1585,6 +1793,7 @@ static int platform_update_schedule(int run_maintenance, int fd)
 		fclose(cron_list);
 	return result;
 }
+#endif
 
 static int update_background_schedule(int run_maintenance)
 {
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1..f0210aa206 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
+	echo "#!/bin/sh\necho \$@ >>args" >print-args &&
+	chmod a+x print-args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	# ~/Library/LaunchAgents
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$UID $PLIST" >>expect &&
+		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args"  git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	# stop does not remove plist files, but boots them out
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$UID $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..620ffbf3af 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '
 	test -z "$GIT_TEST_SKIP_REBASE_P"
 '
 
+test_lazy_prereq MACOS_MAINTENANCE '
+	launchctl list
+'
+
 # Ensure that no test accidentally triggers a Git command
 # that runs 'crontab', affecting a user's cron schedule.
 # Tests that verify the cron integration must set this locally
-- 
gitgitgadget

Re: [PATCH 2/3] maintenance: use launchctl on macOS

[Eric Sunshine]

On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> maintenance: use launchctl on macOS

A few comments below (not necessarily worth a re-roll)...

> The launchctl command needs to be aligned with a user id in order
> to initialize the command environment. This must be done using
> the 'launchctl bootstrap' subcommand. This subcommand is new as
> of macOS 10.11, which was released in September 2015. Before that
> release the 'launchctl load' subcommand was recommended. The best
> source of information on this transition I have seen is available
> at [2].

It's not clear whether or not this is saying that git-maintenance will
dynamically adapt to work with modern and older 'launchctl'. A glance
at the actual code reveals that it knows only about modern
'bootstrap'. Perhaps this could be a bit clearer by saying that it
only supports modern versions, and that support for older versions can
be added later if needed. (For those of us who are stuck with 10-20
year old hardware and OS versions, 2015 isn't that long ago.)

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)
> +static int bootout(const char *filename)
> +{
> +       int result;
> +       struct strvec args = STRVEC_INIT;
> +       char *uid = get_uid();
> +       const char *launchctl = getenv("GIT_TEST_CRONTAB");
> +       if (!launchctl)
> +               launchctl = "/bin/launchctl";
> +
> +       strvec_split(&args, launchctl);
> +       strvec_push(&args, "bootout");
> +       strvec_pushf(&args, "gui/%s", uid);
> +       strvec_push(&args, filename);
> +
> +       result = run_command_v_opt(args.v, 0);
> +
> +       strvec_clear(&args);
> +       free(uid);
> +       return result;
> +}
> +
> +static int bootstrap(const char *filename)
> +{
> +       int result;
> +       struct strvec args = STRVEC_INIT;
> +       char *uid = get_uid();
> +       const char *launchctl = getenv("GIT_TEST_CRONTAB");
> +       if (!launchctl)
> +               launchctl = "/bin/launchctl";
> +
> +       strvec_split(&args, launchctl);
> +       strvec_push(&args, "bootstrap");
> +       strvec_pushf(&args, "gui/%s", uid);
> +       strvec_push(&args, filename);
> +
> +       result = run_command_v_opt(args.v, 0);
> +
> +       strvec_clear(&args);
> +       free(uid);
> +       return result;
> +}

The bootout() and bootstrap() functions seem to be identical except
for one string literal. Code could be reduced by refactoring and
passing that string literal in as an argument.

> +static int remove_plist(enum schedule_priority schedule)
> +{
> +       const char *frequency = get_frequency(schedule);
> +       char *name = get_service_name(frequency);
> +       char *filename = get_service_filename(name);
> +       int result = bootout(filename);
> +       free(filename);
> +       free(name);
> +       return result;
> +}

The result of get_service_name() is only ever passed to
get_service_filename(). If get_service_filename() made the call to
get_service_name() itself, it would free up callers from having to
remember to free(name), thus reducing the likelihood of a possible
leak.

> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
> +{
> +       plist = fopen(filename, "w");
> +
> +       if (!plist)
> +               die(_("failed to open '%s'"), filename);

You can replace the fopen() and die() with a single call to xfopen().

> +       /* bootout might fail if not already running, so ignore */
> +       bootout(filename);
> +       if (bootstrap(filename))
> +               die(_("failed to bootstrap service %s"), filename);

I'm guessing that 'launchctl bootout' won't print a confusing and
unexpected error message if the plist is not presently registered?

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '
> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +       echo "#!/bin/sh\necho \$@ >>args" >print-args &&
> +       chmod a+x print-args &&

Note that $@ loses its special magic if not surrounded by quotes, thus
acts just like $*. So, either use "$@" or $* depending upon your
requirements, but in the case of 'echo', it's just not going to matter
at all, so $* is fine.

To construct the script, you can do this instead, which is easier to
read and handles the 'chmod' for you:

    write_script print-args <<-\EOF
    echo $* >>args
    EOF

> +       for frequency in hourly daily weekly
> +       do
> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> +               grep schedule=$frequency "$PLIST" &&
> +               echo "bootout gui/$UID $PLIST" >>expect &&
> +               echo "bootstrap gui/$UID $PLIST" >>expect || return 1
> +       done &&

My gut feeling is that this would be more robust if you manually
determine UID in the test script the same way as the git-maintenance
command itself does using '/usr/bin/id -u' rather than relying upon
inheriting UID from the user's environment.

> +       # stop does not remove plist files, but boots them out

Is this desirable? Should git-maintenance do a better job of cleaning
up after itself?

Re: [PATCH 2/3] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/3/2020 1:45 PM, Eric Sunshine wrote:
> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> maintenance: use launchctl on macOS
> 
> A few comments below (not necessarily worth a re-roll)...
> 
>> The launchctl command needs to be aligned with a user id in order
>> to initialize the command environment. This must be done using
>> the 'launchctl bootstrap' subcommand. This subcommand is new as
>> of macOS 10.11, which was released in September 2015. Before that
>> release the 'launchctl load' subcommand was recommended. The best
>> source of information on this transition I have seen is available
>> at [2].
> 
> It's not clear whether or not this is saying that git-maintenance will
> dynamically adapt to work with modern and older 'launchctl'. A glance
> at the actual code reveals that it knows only about modern
> 'bootstrap'. Perhaps this could be a bit clearer by saying that it
> only supports modern versions, and that support for older versions can
> be added later if needed. (For those of us who are stuck with 10-20
> year old hardware and OS versions, 2015 isn't that long ago.)

Yes, this is a strange place to be. How far do we go back to support
as many users as possible? How many users will be simultaneously
stuck on an old version of macOS _and_ interested in updating to this
latest version of Git? Is that worth the extra functionality to detect
the the OS version and change commands?

The good news is that this patch doesn't lock us in to the
boot(strap|out) subcommands too much. We could add in load/unload
subcommands for systems that are too old. However, I did think it
was prudent to take the currently-recommended option for fear that
Apple will completely _delete_ the load/unload options in an
upcoming release.

This makes me realize that I should update the documentation to give
pointers for how to view the schedules for each platform:

- Windows: Open "Task Scheduler"
- macOS: 'launchctl list | grep org.git-scm.git'
- Others: 'crontab -l'

>> +static int bootstrap(const char *filename)
>> +{
>> +       int result;
>> +       struct strvec args = STRVEC_INIT;
>> +       char *uid = get_uid();
>> +       const char *launchctl = getenv("GIT_TEST_CRONTAB");
>> +       if (!launchctl)
>> +               launchctl = "/bin/launchctl";
>> +
>> +       strvec_split(&args, launchctl);
>> +       strvec_push(&args, "bootstrap");
>> +       strvec_pushf(&args, "gui/%s", uid);
>> +       strvec_push(&args, filename);
>> +
>> +       result = run_command_v_opt(args.v, 0);
>> +
>> +       strvec_clear(&args);
>> +       free(uid);
>> +       return result;
>> +}
> 
> The bootout() and bootstrap() functions seem to be identical except
> for one string literal. Code could be reduced by refactoring and
> passing that string literal in as an argument.

Good point. Or a simple boolean value for "add" or "remove".
 
>> +static int remove_plist(enum schedule_priority schedule)
>> +{
>> +       const char *frequency = get_frequency(schedule);
>> +       char *name = get_service_name(frequency);
>> +       char *filename = get_service_filename(name);
>> +       int result = bootout(filename);
>> +       free(filename);
>> +       free(name);
>> +       return result;
>> +}
> 
> The result of get_service_name() is only ever passed to
> get_service_filename(). If get_service_filename() made the call to
> get_service_name() itself, it would free up callers from having to
> remember to free(name), thus reducing the likelihood of a possible
> leak.

You're right. In an earlier version I thought I needed to add the
name in the XML, but it turns out I did not.

>> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
>> +{
>> +       plist = fopen(filename, "w");
>> +
>> +       if (!plist)
>> +               die(_("failed to open '%s'"), filename);
> 
> You can replace the fopen() and die() with a single call to xfopen().

Thanks! I'll use that in several places and try to remember next time.

>> +       /* bootout might fail if not already running, so ignore */
>> +       bootout(filename);
>> +       if (bootstrap(filename))
>> +               die(_("failed to bootstrap service %s"), filename);
> 
> I'm guessing that 'launchctl bootout' won't print a confusing and
> unexpected error message if the plist is not presently registered?

You're right, it does. It also returns with a non-zero exit code.
Along with your later suggestion to clear the .plist files, we will
want to have several conditions to not error out during a case where
the task is not scheduled.

>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -389,12 +389,58 @@ test_expect_success 'stop from existing schedule' '
>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +       echo "#!/bin/sh\necho \$@ >>args" >print-args &&
>> +       chmod a+x print-args &&
> 
> Note that $@ loses its special magic if not surrounded by quotes, thus
> acts just like $*. So, either use "$@" or $* depending upon your
> requirements, but in the case of 'echo', it's just not going to matter
> at all, so $* is fine.
> 
> To construct the script, you can do this instead, which is easier to
> read and handles the 'chmod' for you:
> 
>     write_script print-args <<-\EOF
>     echo $* >>args
>     EOF

TIL. thanks.

>> +       for frequency in hourly daily weekly
>> +       do
>> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
>> +               grep schedule=$frequency "$PLIST" &&
>> +               echo "bootout gui/$UID $PLIST" >>expect &&
>> +               echo "bootstrap gui/$UID $PLIST" >>expect || return 1
>> +       done &&
> 
> My gut feeling is that this would be more robust if you manually
> determine UID in the test script the same way as the git-maintenance
> command itself does using '/usr/bin/id -u' rather than relying upon
> inheriting UID from the user's environment.

Yeah, you're right. Thanks!

>> +       # stop does not remove plist files, but boots them out
> 
> Is this desirable? Should git-maintenance do a better job of cleaning
> up after itself?

Yes, let's clear up these .plist files.

Thanks!
-Stolee

Re: [PATCH 2/3] maintenance: use launchctl on macOS

[Eric Sunshine]

On Tue, Nov 3, 2020 at 4:22 PM Derrick Stolee <stolee@gmail.com> wrote:
> On 11/3/2020 1:45 PM, Eric Sunshine wrote:
> > It's not clear whether or not this is saying that git-maintenance will
> > dynamically adapt to work with modern and older 'launchctl'. A glance
> > at the actual code reveals that it knows only about modern
> > 'bootstrap'. Perhaps this could be a bit clearer by saying that it
> > only supports modern versions, and that support for older versions can
> > be added later if needed. (For those of us who are stuck with 10-20
> > year old hardware and OS versions, 2015 isn't that long ago.)
>
> Yes, this is a strange place to be. How far do we go back to support
> as many users as possible? How many users will be simultaneously
> stuck on an old version of macOS _and_ interested in updating to this
> latest version of Git? Is that worth the extra functionality to detect
> the the OS version and change commands?

I don't think this patch series needs to answer these questions
provided that it doesn't close the door to someone adding
older-version support down the road. My review comment was more about
the commit message being clearer about the choice -- supporting only
recent 'launchctl' -- being made by this series. (And perhaps the
documentation could mention that it requires a reasonably modern
'launchctl'.)

> This makes me realize that I should update the documentation to give
> pointers for how to view the schedules for each platform:
>
> - Windows: Open "Task Scheduler"
> - macOS: 'launchctl list | grep org.git-scm.git'
> - Others: 'crontab -l'

Good idea.

I haven't looked at the documentation, but if it doesn't already, I
wonder if it should give examples of how to set these up by hand or
how to customize the ones created by git-maintenance itself. I was
also wondering if git-maintenance could have a mode in which it
generates the template file(s) for you but doesn't actually
activate/install it, instead providing instructions for
activation/installation. That way, people could modify the scheduling
file before actually activating it. However, this may all be outside
the scope of the patch series, and could be done later if desired.

Re: [PATCH 2/3] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/3/2020 5:27 PM, Eric Sunshine wrote:
> On Tue, Nov 3, 2020 at 4:22 PM Derrick Stolee <stolee@gmail.com> wrote:
>> On 11/3/2020 1:45 PM, Eric Sunshine wrote:
>>> It's not clear whether or not this is saying that git-maintenance will
>>> dynamically adapt to work with modern and older 'launchctl'. A glance
>>> at the actual code reveals that it knows only about modern
>>> 'bootstrap'. Perhaps this could be a bit clearer by saying that it
>>> only supports modern versions, and that support for older versions can
>>> be added later if needed. (For those of us who are stuck with 10-20
>>> year old hardware and OS versions, 2015 isn't that long ago.)
>>
>> Yes, this is a strange place to be. How far do we go back to support
>> as many users as possible? How many users will be simultaneously
>> stuck on an old version of macOS _and_ interested in updating to this
>> latest version of Git? Is that worth the extra functionality to detect
>> the the OS version and change commands?
> 
> I don't think this patch series needs to answer these questions
> provided that it doesn't close the door to someone adding
> older-version support down the road. My review comment was more about
> the commit message being clearer about the choice -- supporting only
> recent 'launchctl' -- being made by this series. (And perhaps the
> documentation could mention that it requires a reasonably modern
> 'launchctl'.)

Thanks. I'll be sure to make the commit message more clear.
 
>> This makes me realize that I should update the documentation to give
>> pointers for how to view the schedules for each platform:
>>
>> - Windows: Open "Task Scheduler"
>> - macOS: 'launchctl list | grep org.git-scm.git'
>> - Others: 'crontab -l'
> 
> Good idea.
> 
> I haven't looked at the documentation, but if it doesn't already, I
> wonder if it should give examples of how to set these up by hand or
> how to customize the ones created by git-maintenance itself. I was
> also wondering if git-maintenance could have a mode in which it
> generates the template file(s) for you but doesn't actually
> activate/install it, instead providing instructions for
> activation/installation. That way, people could modify the scheduling
> file before actually activating it. However, this may all be outside
> the scope of the patch series, and could be done later if desired.

Outside of the technical details, the biggest questions I've
tried to handle with the background maintenance feature has
been to balance customization with ease-of-use. My philosophy
is that users fall into a few expertise buckets, and have
different expectations:

* Beginners will never know about background maintenance and
  so will never run "git maintenance start" or set the config
  values.

* Intermediate users might discover "git maintenance start"
  and will appreciate that they don't need to learn cron to
  set up a good default schedule.

* Advanced users will read the documentation and use Git
  config settings to customize their maintenance tasks and
  schedule.

* Expert users might decide the task schedule available by
  "git maintenance start" is too restrictive, so they create
  their own background schedule with custom tasks. They might
  not even run the maintenance builtin and opt instead for
  'git gc' or 'git repack' directly!

My main target has been these "intermediate" users who might
run the command and forget about it. However, I've also tried
to keep the advanced users in mind with the advanced config
options available.

Your comment about documentation demonstrates a way to serve
the advanced and expert users by providing a clear framework
for discovering what Git is doing under the hood and how to
modify or adapt that to their needs. It is also important to
demonstrate how to set up schedules in a similar way without
having them be overwritten by a later "git maintenance start"
command.

I will give this a shot in v2. Thanks.

-Stolee

Re: [PATCH 2/3] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/3/2020 4:21 PM, Derrick Stolee wrote:
> On 11/3/2020 1:45 PM, Eric Sunshine wrote:
>> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget
>>> +static int remove_plist(enum schedule_priority schedule)
>>> +{
>>> +       const char *frequency = get_frequency(schedule);
>>> +       char *name = get_service_name(frequency);
>>> +       char *filename = get_service_filename(name);
>>> +       int result = bootout(filename);
>>> +       free(filename);
>>> +       free(name);
>>> +       return result;
>>> +}
>>
>> The result of get_service_name() is only ever passed to
>> get_service_filename(). If get_service_filename() made the call to
>> get_service_name() itself, it would free up callers from having to
>> remember to free(name), thus reducing the likelihood of a possible
>> leak.
> 
> You're right. In an earlier version I thought I needed to add the
> name in the XML, but it turns out I did not.

As I go through the effort to remove get_service_name() I find that
actually the name is used in one place in the XML file:

+		   "<key>Label</key><string>%s</string>\n"

This "Label" should match the filename, I believe. I can still
be more careful about how often this name is actually required.

Thanks,
-Stolee

[PATCH 3/3] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 181 +++++++++++++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh |  40 ++++++++-
 2 files changed, 218 insertions(+), 3 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index fa0ae63a80..24511fec2e 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)
 	else
 		return remove_plists();
 }
+
+#elif defined(GIT_WINDOWS_NATIVE)
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int remove_task(enum schedule_priority schedule)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+	const char *schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+
+	strvec_split(&args, schtasks);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int remove_scheduled_tasks(void)
+{
+	return remove_task(SCHEDULE_HOURLY) ||
+		remove_task(SCHEDULE_DAILY) ||
+		remove_task(SCHEDULE_WEEKLY);
+}
+
+static int schedule_task(const char *exec_path, enum schedule_priority schedule)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *xml, *schtasks;
+	char *xmlpath;
+	FILE *xmlfp;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+
+	xmlpath =  xstrfmt("%s/schedule-%s.xml",
+			   the_repository->objects->odb->path,
+			   frequency);
+	xmlfp = fopen(xmlpath, "w");
+	if (!xmlfp)
+		die(_("failed to open '%s'"), xmlpath);
+
+	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fprintf(xmlfp, xml);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml=  "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(xmlfp, xml, exec_path, exec_path, frequency);
+	fclose(xmlfp);
+
+	schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+	strvec_split(&args, schtasks);
+	strvec_pushl(&args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	unlink(xmlpath);
+	free(xmlpath);
+	free(name);
+	return result;
+}
+
+static int add_scheduled_tasks(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schedule_task(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_scheduled_tasks();
+	else
+		return remove_scheduled_tasks();
+}
+
 #else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index f0210aa206..73dc0078da 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
@@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success MINGW 'start and stop Windows maintenance' '
+	echo "echo \$@ >>args" >print-args &&
+	chmod a+x print-args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
+	cat args &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \
+			|| return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args"  git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		echo "/delete /tn Git Maintenance ($frequency) /f" >>expect \
+			|| return 1
+	done &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH 3/3] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> There is a deficiency in the current design. Windows has two kinds of
> applications: GUI applications that start by "winmain()" and console
> applications that start by "main()". Console applications are attached
> to a new Console window if they are not already associated with a GUI
> application. This means that every hour the scheudled task launches a
> command window for the scheduled tasks. Not only is this visually
> obtrusive, but it also takes focus from whatever else the user is
> doing!

I wonder if you could use the technique explained in [1] to prevent
the console window from popping up.

[1]: https://pureinfotech.com/prevent-command-window-appearing-scheduled-tasks-windows-10/

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +test_expect_success MINGW 'start and stop Windows maintenance' '
> +       echo "echo \$@ >>args" >print-args &&
> +       chmod a+x print-args &&

Same comments as my review of [2/3] regarding $@ and write_script().

> +       rm -f args &&
> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
> +       cat args &&

Is this 'cat' leftover debugging gunk?

> +       # start registers the repo
> +       git config --get --global maintenance.repo "$(pwd)" &&
> +
> +       rm expect &&
> +       for frequency in hourly daily weekly
> +       do
> +               echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \
> +                       || return 1
> +       done &&

Rather than using >> within the loop, it's often simpler to capture
the output of the for-loop in its entirety:

    for frequency in hourly daily weekly
    do
        echo "/create ..." || return 1
    done >expect &&

However, in this case 'printf' may be even simpler:

    printf "/create /tn ... .git/objects/schedule-%s.xml\n" \
        hourly daily weekly >expect &&

> +       GIT_TEST_CRONTAB="/bin/sh print-args"  git maintenance stop &&

Too many spaces before the 'git' command.

Re: [PATCH 3/3] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/3/2020 2:06 PM, Eric Sunshine wrote:
> On Tue, Nov 3, 2020 at 9:05 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> There is a deficiency in the current design. Windows has two kinds of
>> applications: GUI applications that start by "winmain()" and console
>> applications that start by "main()". Console applications are attached
>> to a new Console window if they are not already associated with a GUI
>> application. This means that every hour the scheudled task launches a
>> command window for the scheduled tasks. Not only is this visually
>> obtrusive, but it also takes focus from whatever else the user is
>> doing!
> 
> I wonder if you could use the technique explained in [1] to prevent
> the console window from popping up.
> 
> [1]: https://pureinfotech.com/prevent-command-window-appearing-scheduled-tasks-windows-10/

The critical part of that strategy is the "Run whether the user is
logged in or not". The resulting option that triggers causes the
schtasks command to require a password prompt (or a password passed
as a command-line argument). I found that interaction to be too
disruptive.

>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -441,6 +441,40 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +test_expect_success MINGW 'start and stop Windows maintenance' '
>> +       echo "echo \$@ >>args" >print-args &&
>> +       chmod a+x print-args &&
> 
> Same comments as my review of [2/3] regarding $@ and write_script().

Noted!

>> +       rm -f args &&
>> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
>> +       cat args &&
> 
> Is this 'cat' leftover debugging gunk?

Yes. Thanks.

>> +       # start registers the repo
>> +       git config --get --global maintenance.repo "$(pwd)" &&
>> +
>> +       rm expect &&
>> +       for frequency in hourly daily weekly
>> +       do
>> +               echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \
>> +                       || return 1
>> +       done &&
> 
> Rather than using >> within the loop, it's often simpler to capture
> the output of the for-loop in its entirety:
> 
>     for frequency in hourly daily weekly
>     do
>         echo "/create ..." || return 1
>     done >expect &&
> 
> However, in this case 'printf' may be even simpler:
> 
>     printf "/create /tn ... .git/objects/schedule-%s.xml\n" \
>         hourly daily weekly >expect &&

Excellent.

>> +       GIT_TEST_CRONTAB="/bin/sh print-args"  git maintenance stop &&
> 
> Too many spaces before the 'git' command.

Thanks!
-Stolee

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This is based on ds/maintenance-part-3.
>
> After sitting with the background maintenance as it has been cooking, I
> wanted to come back around and implement the background maintenance for
> Windows. However, I noticed that there were some things bothering me with
> background maintenance on my macOS machine. These are detailed in PATCH 2,
> but the tl;dr is that 'cron' is not recommended by Apple and instead
> 'launchd' satisfies our needs.

Nicely done.

> This series implements the background scheduling so git maintenance
> (start|stop) works on those platforms. I've been operating with these
> schedules for a while now without the problems described in the patches.
>
> There is a particularly annoying case about console windows popping up on
> Windows, but PATCH 3 describes a plan to get around that.
>
> Thanks, -Stolee
>
> Derrick Stolee (3):
>   maintenance: extract platform-specific scheduling
>   maintenance: use launchctl on macOS
>   maintenance: use Windows scheduled tasks
>
>  builtin/gc.c           | 428 +++++++++++++++++++++++++++++++++++++++--
>  t/t7900-maintenance.sh |  86 ++++++++-
>  t/test-lib.sh          |   4 +
>  3 files changed, 498 insertions(+), 20 deletions(-)
>
>
> base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v1
> Pull-Request: https://github.com/gitgitgadget/git/pull/776

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This is based on ds/maintenance-part-3.

Ah, I forgot to ask those on CC list how carefully they read the
part 3 of the series, as it's been left on 'seen' for some time, and
I do not know if it is ready to start cooking in 'next'.

Thanks.

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 11/3/2020 3:21 PM, Junio C Hamano wrote:
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> This is based on ds/maintenance-part-3.
> 
> Ah, I forgot to ask those on CC list how carefully they read the
> part 3 of the series, as it's been left on 'seen' for some time, and
> I do not know if it is ready to start cooking in 'next'.

It has been a while since anyone has commented, and I've been
running the patches locally for quite a while. I'd be very
happy to see them cook in next. I wasn't quite to the place
to be pushy about it.

I'm hoping that parts 3 and 4 can make it in time for 2.30,
so the feature is universally available for all platforms.
I realize that's not entirely up to just me.

Thanks,
-Stolee

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

Derrick Stolee <stolee@gmail.com> writes:

> On 11/3/2020 3:21 PM, Junio C Hamano wrote:
>> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> 
>>> This is based on ds/maintenance-part-3.
>> 
>> Ah, I forgot to ask those on CC list how carefully they read the
>> part 3 of the series, as it's been left on 'seen' for some time, and
>> I do not know if it is ready to start cooking in 'next'.
>
> It has been a while since anyone has commented, and I've been
> running the patches locally for quite a while. I'd be very
> happy to see them cook in next. I wasn't quite to the place
> to be pushy about it.
>
> I'm hoping that parts 3 and 4 can make it in time for 2.30,
> so the feature is universally available for all platforms.
> I realize that's not entirely up to just me.

After writing this entry in the What's cooking report:

    * ds/maintenance-part-4 (2020-11-03) 3 commits
     - maintenance: use Windows scheduled tasks
     - maintenance: use launchctl on macOS
     - maintenance: extract platform-specific scheduling
     (this branch uses ds/maintenance-part-3.)

     Follow-up on the "maintenance part-3" which introduced scheduled
     maintenance tasks to support platforms whose native scheduling
     methods are not 'cron'.

I was wondering if I should propose making these two parts into one,
so we may be pretty much on the same page.

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 11/3/2020 5:30 PM, Junio C Hamano wrote:
> Derrick Stolee <stolee@gmail.com> writes:
>> I'm hoping that parts 3 and 4 can make it in time for 2.30,
>> so the feature is universally available for all platforms.
>> I realize that's not entirely up to just me.
> 
> After writing this entry in the What's cooking report:
> 
>     * ds/maintenance-part-4 (2020-11-03) 3 commits
>      - maintenance: use Windows scheduled tasks
>      - maintenance: use launchctl on macOS
>      - maintenance: extract platform-specific scheduling
>      (this branch uses ds/maintenance-part-3.)
> 
>      Follow-up on the "maintenance part-3" which introduced scheduled
>      maintenance tasks to support platforms whose native scheduling
>      methods are not 'cron'.

This is a good summary of this series.

> I was wondering if I should propose making these two parts into one,
> so we may be pretty much on the same page.

I'm happy to have the two topics be merged into one series, but
I'd prefer to only re-roll these three patches during review.
Of course, by keeping them together we have the ability to re-
roll all of the patches, but part-3 has a decent length and has
been stable for a while.

I defer to what is easiest for you.

Thanks,
-Stolee

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

Derrick Stolee <stolee@gmail.com> writes:

> On 11/3/2020 5:30 PM, Junio C Hamano wrote:
>> Derrick Stolee <stolee@gmail.com> writes:
>>> I'm hoping that parts 3 and 4 can make it in time for 2.30,
>>> so the feature is universally available for all platforms.
>>> I realize that's not entirely up to just me.
>> ...
>> I was wondering if I should propose making these two parts into one,
>> so we may be pretty much on the same page.
>
> I'm happy to have the two topics be merged into one series, but
> I'd prefer to only re-roll these three patches during review.
> Of course, by keeping them together we have the ability to re-
> roll all of the patches, but part-3 has a decent length and has
> been stable for a while.
>
> I defer to what is easiest for you.

Two topics, one on top of the other, is fine, as long as I can
remember (and you help me by reminding) to keep them together,
and the bottom one is reasonably solid that I do not have to do the
rebasing myself ;-)

Re: [PATCH 0/3] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 11/4/2020 12:00 PM, Junio C Hamano wrote:
> Derrick Stolee <stolee@gmail.com> writes:
> 
>> On 11/3/2020 5:30 PM, Junio C Hamano wrote:
>>> Derrick Stolee <stolee@gmail.com> writes:
>>>> I'm hoping that parts 3 and 4 can make it in time for 2.30,
>>>> so the feature is universally available for all platforms.
>>>> I realize that's not entirely up to just me.
>>> ...
>>> I was wondering if I should propose making these two parts into one,
>>> so we may be pretty much on the same page.
>>
>> I'm happy to have the two topics be merged into one series, but
>> I'd prefer to only re-roll these three patches during review.
>> Of course, by keeping them together we have the ability to re-
>> roll all of the patches, but part-3 has a decent length and has
>> been stable for a while.
>>
>> I defer to what is easiest for you.
> 
> Two topics, one on top of the other, is fine, as long as I can
> remember (and you help me by reminding) to keep them together,
> and the bottom one is reasonably solid that I do not have to do the
> rebasing myself ;-)

Sounds good to me. If I need to re-roll part 3, then
I'll consider replacing both parts with a new series.

Thanks,
-Stolee

[PATCH v2 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.

Updates in V2
=============

 * This is a faster turnaround for a v2 than I would normally like, but Eric
   inspired extra documentation about how to customize background schedules.
   
   
 * New extensions to git-maintenance.txt include guidelines for inspecting
   what git maintenance start does and how to customize beyond that. This
   includes a new PATCH 2 that includes documentation for 'cron' on
   non-macOS non-Windows systems.
   
   
 * Several improvements, especially in the tests, are included.
   
   
 * While testing manually, I noticed that somehow I had incorrectly had an
   opening <dict> tag instead of a closing </dict> tag in the hourly format
   on macOS. I found that the xmllint tool can verify the XML format of a
   file, which catches the bug. This seems like a good approach since the
   test is macOS-only. Does anyone have concerns about adding this
   dependency?
   
   

Thanks, -Stolee

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 119 +++++++++
 builtin/gc.c                      | 428 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            |  83 +++++-
 t/test-lib.sh                     |   4 +
 4 files changed, 614 insertions(+), 20 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v1:

 1:  d35f1aa162 = 1:  d35f1aa162 maintenance: extract platform-specific scheduling
 -:  ---------- > 2:  709a173720 maintenance: include 'cron' details in docs
 2:  832fdf1687 ! 3:  0fafd75d10 maintenance: use launchctl on macOS
     @@ Commit message
          and 'git maintenance stop' by injecting a script that writes the
          command-line arguments into GIT_TEST_CRONTAB.
      
     +    An earlier version of this patch accidentally had an opening
     +    "<dict>" tag when it should have had a closing "</dict>" tag. This
     +    was caught during manual testing with actual 'launchctl' commands,
     +    but we do not want to update developers' tasks when running tests.
     +    It appears that macOS includes the "xmllint" tool which can verify
     +    the XML format, so call it from the macOS-specific tests to ensure
     +    the .plist files are well-formatted.
     +
     +    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
     + ## Documentation/git-maintenance.txt ##
     +@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
     + executing the correct binaries in your schedule.
     + 
     + 
     ++BACKGROUND MAINTENANCE ON MACOS SYSTEMS
     ++---------------------------------------
     ++
     ++While macOS technically supports `cron`, using `crontab -e` requires
     ++elevated privileges and the executed process do not have a full user
     ++context. Without a full user context, Git and its credential helpers
     ++cannot access stored credentials, so some maintenance tasks are not
     ++functional.
     ++
     ++Instead, `git maintenance start` interacts with the `launchctl` tool,
     ++which is the recommended way to
     ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].
     ++
     ++Scheduling maintenance through `git maintenance (start|stop)` requires
     ++some `launchctl` features available only in macOS 10.11 or later.
     ++
     ++Your user-specific scheduled tasks are stored as XML-formatted `.plist`
     ++files in `~/Library/LaunchAgents/`. You can see the currently-registered
     ++tasks using the following command:
     ++
     ++-----------------------------------------------------------------------
     ++$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git
     ++org.git-scm.git.daily.plist
     ++org.git-scm.git.hourly.plist
     ++org.git-scm.git.weekly.plist
     ++-----------------------------------------------------------------------
     ++
     ++One task is registered for each `--schedule=<frequency>` option. To
     ++inspect how the XML format describes each schedule, open one of these
     ++`.plist` files in an editor and inspect the `<array>` element following
     ++the `<key>StartCalendarInterval</key>` element.
     ++
     ++`git maintenance start` will overwrite these files and register the
     ++tasks again with `launchctl`, so any customizations should be done by
     ++creating your own `.plist` files with distinct names. Similarly, the
     ++`git maintenance stop` command will unregister the tasks with `launchctl`
     ++and delete the `.plist` files.
     ++
     ++To create more advanced customizations to your background tasks, see
     ++https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
     ++for more information.
     ++
     ++
     + GIT
     + ---
     + Part of the linkgit:git[1] suite
     +
       ## builtin/gc.c ##
      @@ builtin/gc.c: static int maintenance_unregister(void)
       	return run_command(&config_unset);
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +		repeat = "<dict>\n"
      +			 "<key>Hour</key><integer>%d</integer>\n"
      +			 "<key>Minute</key><integer>0</integer>\n"
     -+			 "<dict>\n";
     ++			 "</dict>\n";
      +		for (i = 1; i <= 23; i++)
      +			fprintf(plist, repeat, i);
      +		break;
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
      +	for frequency in hourly daily weekly
      +	do
      +		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
     ++		xmllint "$PLIST" >/dev/null &&
      +		grep schedule=$frequency "$PLIST" &&
      +		echo "bootout gui/$UID $PLIST" >>expect &&
      +		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
 3:  a9221cc4aa ! 4:  84eb44de31 maintenance: use Windows scheduled tasks
     @@ Commit message
          short term. In the long term, we can consider creating this GUI
          shim application within core Git, perhaps in contrib/.
      
     +    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
     + ## Documentation/git-maintenance.txt ##
     +@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy
     + for more information.
     + 
     + 
     ++BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
     ++-----------------------------------------
     ++
     ++Windows does not support `cron` and instead has its own system for
     ++scheduling background tasks. The `git maintenance start` command uses
     ++the `schtasks` command to submit tasks to this system. You can inspect
     ++all background tasks using the Task Scheduler application. The tasks
     ++added by Git have names of the form `Git Maintenance (<frequency>)`.
     ++The Task Scheduler GUI has ways to inspect these tasks, but you can also
     ++export the tasks to XML files and view the details there.
     ++
     ++Note that since Git is a console application, these background tasks
     ++create a console window visible to the current user. This can be changed
     ++manually by selecting the "Run whether user is logged in or not" option
     ++in Task Scheduler. This change requires a password input, which is why
     ++`git maintenance start` does not select it by default.
     ++
     ++If you want to customize the background tasks, please rename the tasks
     ++so future calls to `git maintenance (start|stop)` do not overwrite your
     ++custom tasks.
     ++
     ++
     + GIT
     + ---
     + Part of the linkgit:git[1] suite
     +
       ## builtin/gc.c ##
      @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
       	else
     @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma
       '
       
      +test_expect_success MINGW 'start and stop Windows maintenance' '
     -+	echo "echo \$@ >>args" >print-args &&
     -+	chmod a+x print-args &&
     ++	write_script print-args <<-\EOF &&
     ++	echo $* >>args
     ++	EOF
      +
      +	rm -f args &&
      +	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
     -+	cat args &&
      +
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	rm expect &&
      +	for frequency in hourly daily weekly
      +	do
     -+		echo "/create /tn Git Maintenance ($frequency) /f /xml .git/objects/schedule-$frequency.xml" >>expect \
     -+			|| return 1
     -+	done &&
     ++		printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
     ++			$frequency $frequency
     ++	done >expect &&
      +	test_cmp expect args &&
      +
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="/bin/sh print-args"  git maintenance stop &&
     ++	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
      +
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
      +	rm expect &&
     -+	for frequency in hourly daily weekly
     -+	do
     -+		echo "/delete /tn Git Maintenance ($frequency) /f" >>expect \
     -+			|| return 1
     -+	done &&
     ++	printf "/delete /tn Git Maintenance (%s) /f\n" \
     ++		hourly daily weekly >expect &&
      +	test_cmp expect args
      +'
      +

-- 
gitgitgadget

[PATCH v2 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c | 38 +++++++++++++++++++++-----------------
 1 file changed, 21 insertions(+), 17 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a1..c1f7d9bdc2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int platform_update_schedule(int run_maintenance, int fd)
 {
 	int result = 0;
 	int in_old_region = 0;
@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)
 	FILE *cron_list, *cron_in;
 	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
-
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
 
 	crontab_name = getenv("GIT_TEST_CRONTAB");
 	if (!crontab_name)
@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)
 	strvec_split(&crontab_list.args, crontab_name);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
 	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 	}
 
 	/* Ignore exit code, as an empty crontab will return error. */
@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
 	strvec_split(&crontab_edit.args, crontab_name);
@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)
 	crontab_edit.git_cmd = 0;
 
 	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 	}
 
 	cron_in = fdopen(crontab_edit.in, "w");
@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
-	}
-	fclose(cron_list);
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static int update_background_schedule(int run_maintenance)
+{
+	int result;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	result = platform_update_schedule(run_maintenance, lk.tempfile->fd);
 
-cleanup:
 	rollback_lock_file(&lk);
 	return result;
 }
-- 
gitgitgadget

[PATCH v2 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc..4c7aac877d 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is `cron`. This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `<path>` string is loaded to specifically use the location for the
+`git` executable used in the `git maintenance start` command. This allows
+for multiple versions to be compatible. However, if the same user runs
+`git maintenance start` with multiple Git executables, then only the
+latest executable will be used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config located at `~/.gitconfig`.
+The `git maintenance` process then determines which maintenance tasks
+are configured to run on each repository with each `<frequency>` using
+the `maintenance.<task>.schedule` config options. These values are loaded
+from the global or repository config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
+for advanced scheduling techniques. Please do use the full path and
+`--exec-path` techniques from the default schedule to ensure you are
+executing the correct binaries in your schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

Re: [PATCH v2 2/4] maintenance: include 'cron' details in docs

[Eric Sunshine]

On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Advanced and expert users may want to know how 'git maintenance start'
> schedules background maintenance in order to customize their own
> schedules beyond what the maintenance.* config values allow. Start a new
> set of sections in git-maintenance.txt that describe how 'cron' is used
> to run these tasks.
>
> This is particularly valuable for users who want to inspect what Git is
> doing or for users who want to customize the schedule further. Having a
> baseline can provide a way forward for users who have never worked with
> cron schedules.

A few comments below, not necessarily worth a re-roll...

> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
> +The standard mechanism for scheduling background tasks on POSIX systems
> +is `cron`. This tool executes commands based on a given schedule. The

It is common in Git (and other Unix) documentation to refer to a Unix
tool by its man page reference. So, for instance, instead of `cron`,
we would say cron(8) since the `cron` man page is in section 8 of the
Unix manual.

> +The `<path>` string is loaded to specifically use the location for the

The word "loaded" sounds odd in this context.

> +`git` executable used in the `git maintenance start` command. This allows
> +for multiple versions to be compatible. However, if the same user runs
> +`git maintenance start` with multiple Git executables, then only the
> +latest executable will be used.

I had to read this paragraph four or five times to understand what it
is trying to say (assuming I do understand it). Perhaps it can be
rewritten more succinctly something like this:

    The `crontab` entry specifies the full path of the `git`
    executable to ensure that the `git` command run is the same one
    with which `git maintenance start` was issued independent of
    `PATH`.

> +These commands use `git for-each-repo --config=maintenance.repo` to run
> +`git maintenance run --schedule=<frequency>` on each repository listed in
> +the multi-valued `maintenance.repo` config option. These are typically
> +loaded from the user-specific global config located at `~/.gitconfig`.

I wonder if mentioning `~/.gitconfig` explicitly is wise since it
might also be at $XDG_CONFIG_HOME/git/config or some other location on
Windows. Perhaps it would be sufficient to mention only "global Git
configuration" or something.

> +If the config values are insufficient to achieve your desired background
> +maintenance schedule, then you can create your own schedule. If you run
> +`crontab -e`, then an editor will load with your user-specific `cron`
> +schedule. In that editor, you can add your own schedule lines. You could
> +start by adapting the default schedule listed earlier, or you could read
> +https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]

For consistency with other parts of Git documentation, it might be
better to reference the `crontab` man page simply as crontab(5) rather
than providing an explicit URL.

> +for advanced scheduling techniques. Please do use the full path and
> +`--exec-path` techniques from the default schedule to ensure you are
> +executing the correct binaries in your schedule.

[PATCH v2 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2].

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_CRONTAB.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format, so call it from the macOS-specific tests to ensure
the .plist files are well-formatted.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  43 ++++++
 builtin/gc.c                      | 209 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  53 +++++++-
 t/test-lib.sh                     |   4 +
 4 files changed, 306 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 4c7aac877d..451ebac131 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,49 @@ for advanced scheduling techniques. Please do use the full path and
 executing the correct binaries in your schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process do not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to
+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].
+
+Scheduling maintenance through `git maintenance (start|stop)` requires
+some `launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
+for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index c1f7d9bdc2..7604064a8d 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+#if defined(__APPLE__)
+
+static char *get_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *get_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_uid(void)
+{
+	struct strbuf output = STRBUF_INIT;
+	struct child_process id = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL);
+	if (capture_command(&id, &output, 0))
+		die(_("failed to discover user id"));
+
+	strbuf_trim_trailing_newline(&output);
+	return strbuf_detach(&output, NULL);
+}
+
+static int bootout(const char *filename)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	char *uid = get_uid();
+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
+	if (!launchctl)
+		launchctl = "/bin/launchctl";
+
+	strvec_split(&args, launchctl);
+	strvec_push(&args, "bootout");
+	strvec_pushf(&args, "gui/%s", uid);
+	strvec_push(&args, filename);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(uid);
+	return result;
+}
+
+static int bootstrap(const char *filename)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	char *uid = get_uid();
+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
+	if (!launchctl)
+		launchctl = "/bin/launchctl";
+
+	strvec_split(&args, launchctl);
+	strvec_push(&args, "bootstrap");
+	strvec_pushf(&args, "gui/%s", uid);
+	strvec_push(&args, filename);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(uid);
+	return result;
+}
+
+static int remove_plist(enum schedule_priority schedule)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+	int result = bootout(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int remove_plists(void)
+{
+	return remove_plist(SCHEDULE_HOURLY) ||
+		remove_plist(SCHEDULE_DAILY) ||
+		remove_plist(SCHEDULE_WEEKLY);
+}
+
+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = fopen(filename, "w");
+
+	if (!plist)
+		die(_("failed to open '%s'"), filename);
+
+	preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+
+	/* bootout might fail if not already running, so ignore */
+	bootout(filename);
+	if (bootstrap(filename))
+		die(_("failed to bootstrap service %s"), filename);
+
+	fclose(plist);
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int add_plists(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		schedule_plist(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_plists();
+	else
+		return remove_plists();
+}
+#else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1585,6 +1793,7 @@ static int platform_update_schedule(int run_maintenance, int fd)
 		fclose(cron_list);
 	return result;
 }
+#endif
 
 static int update_background_schedule(int run_maintenance)
 {
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1..1c43b34a93 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,12 +389,59 @@ test_expect_success 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
+	echo "#!/bin/sh\necho \$@ >>args" >print-args &&
+	chmod a+x print-args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	# ~/Library/LaunchAgents
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		xmllint "$PLIST" >/dev/null &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$UID $PLIST" >>expect &&
+		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args"  git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	# stop does not remove plist files, but boots them out
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$UID $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..620ffbf3af 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '
 	test -z "$GIT_TEST_SKIP_REBASE_P"
 '
 
+test_lazy_prereq MACOS_MAINTENANCE '
+	launchctl list
+'
+
 # Ensure that no test accidentally triggers a Git command
 # that runs 'crontab', affecting a user's cron schedule.
 # Tests that verify the cron integration must set this locally
-- 
gitgitgadget

Re: [PATCH v2 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> [...]
> The solution is to switch from cron to the Apple-recommended [1]
> 'launchd' tool.
> [...]
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> +While macOS technically supports `cron`, using `crontab -e` requires
> +elevated privileges and the executed process do not have a full user

Either s/process/processes/ or s/do/does/

> +context. Without a full user context, Git and its credential helpers
> +cannot access stored credentials, so some maintenance tasks are not
> +functional.

Nicely explained.

> +Instead, `git maintenance start` interacts with the `launchctl` tool,
> +which is the recommended way to
> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].

Nit: I worry a bit about links to Apple documentation becoming
outdated. It might not hurt to omit this link altogether, or perhaps
demote it to a footnote (which might allow it to be somewhat usable
even when Git documentation is rendered into something other than
HTML).

> +Scheduling maintenance through `git maintenance (start|stop)` requires
> +some `launchctl` features available only in macOS 10.11 or later.

Nit: This leaves the reader wondering what modern features are needed.
Would it make sense to mention that "bootstrap" is used in place of
"load" in older versions of 'launchctl'?

> +Your user-specific scheduled tasks are stored as XML-formatted `.plist`
> +files in `~/Library/LaunchAgents/`. You can see the currently-registered
> +tasks using the following command:
> +
> +-----------------------------------------------------------------------
> +$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git

Alternately (unimportant):

    ls ~/Library/LaunchAgents/org.git-scm.git.*

although that would emit "No such file" if you don't have any
registered, which might suggest:

    find ~/Library/LaunchAgents -name 'org.git-scm.git.*'

> +To create more advanced customizations to your background tasks, see
> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
> +for more information.

I really worry about this sort of URL becoming outdated. Would it make
sense instead to just point the user at the man page,
launchd.plist(5)? It's not quite the same, as it doesn't provide the
range of examples as the URL you cite, but it should get the user
started.

> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)
> +static int remove_plist(enum schedule_priority schedule)
> +{
> +       const char *frequency = get_frequency(schedule);
> +       char *name = get_service_name(frequency);
> +       char *filename = get_service_filename(name);
> +       int result = bootout(filename);
> +       free(filename);
> +       free(name);
> +       return result;
> +}
>
> +static int remove_plists(void)
> +{
> +       return remove_plist(SCHEDULE_HOURLY) ||
> +               remove_plist(SCHEDULE_DAILY) ||
> +               remove_plist(SCHEDULE_WEEKLY);
> +}

The new documentation you added says that the plist files will be
deleted after they are deregistered using launchctl, but I don't see
anything actually deleting them. Am I missing something obvious?

> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
> +{
> +       plist = fopen(filename, "w");
> +       if (!plist)
> +               die(_("failed to open '%s'"), filename);

As mentioned previously, these could be replaced with a simple xfopen().

In fact, I'm having trouble seeing changes in this re-roll which you
had planned on making, such as consolidating the repeated code in
bootout() and bootstrap(), and ensuring that bootout() doesn't
complain if the plist files are already missing, and so forth. Did you
opt to not make those changes? (Which would be fine; they were minor
suggestions.)

> +       preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
> +                  "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
> +                  "<plist version=\"1.0\">"
> +                  "<dict>\n"
> +                  "<key>Label</key><string>%s</string>\n"
> +                  "<key>ProgramArguments</key>\n"
> +                  "<array>\n"
> +                  "<string>%s/git</string>\n"
> +                  "<string>--exec-path=%s</string>\n"
> +                  "<string>for-each-repo</string>\n"
> +                  "<string>--config=maintenance.repo</string>\n"
> +                  "<string>maintenance</string>\n"
> +                  "<string>run</string>\n"
> +                  "<string>--schedule=%s</string>\n"
> +                  "</array>\n"
> +                  "<key>StartCalendarInterval</key>\n"
> +                  "<array>\n";
> +       fprintf(plist, preamble, name, exec_path, exec_path, frequency);

The Git test framework ensures that this will be written into the test
directory rather than the user's actual ~/Library/LaunchAgents
directory during testing. Okay.

> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +       echo "#!/bin/sh\necho \$@ >>args" >print-args &&
> +       chmod a+x print-args &&

Earlier review already mentioned write_script() and "$@". (Not
necessarily worth a re-roll.)

> +       for frequency in hourly daily weekly
> +       do
> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> +               xmllint "$PLIST" >/dev/null &&

Do we really need to suppress xmllint's stdout?

> +               grep schedule=$frequency "$PLIST" &&
> +               echo "bootout gui/$UID $PLIST" >>expect &&
> +               echo "bootstrap gui/$UID $PLIST" >>expect || return 1
> +       done &&
> +       test_cmp expect args &&
> +
> +       rm -f args &&
> +       GIT_TEST_CRONTAB="./print-args"  git maintenance stop &&

There is still an extra space between the closing quote and git
command (mentioned previously).

> +       # stop does not unregister the repo
> +       git config --get --global maintenance.repo "$(pwd)" &&
> +
> +       # stop does not remove plist files, but boots them out

Documentation added in this re-roll claims that the plist files do get deleted.

Re: [PATCH v2 3/4] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/11/2020 3:12 AM, Eric Sunshine wrote:
> On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> [...]
>> The solution is to switch from cron to the Apple-recommended [1]
>> 'launchd' tool.
>> [...]
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
>> +While macOS technically supports `cron`, using `crontab -e` requires
>> +elevated privileges and the executed process do not have a full user
> 
> Either s/process/processes/ or s/do/does/
> 
>> +context. Without a full user context, Git and its credential helpers
>> +cannot access stored credentials, so some maintenance tasks are not
>> +functional.
> 
> Nicely explained.
> 
>> +Instead, `git maintenance start` interacts with the `launchctl` tool,
>> +which is the recommended way to
>> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].
> 
> Nit: I worry a bit about links to Apple documentation becoming
> outdated. It might not hurt to omit this link altogether, or perhaps
> demote it to a footnote (which might allow it to be somewhat usable
> even when Git documentation is rendered into something other than
> HTML).
> 
>> +Scheduling maintenance through `git maintenance (start|stop)` requires
>> +some `launchctl` features available only in macOS 10.11 or later.
> 
> Nit: This leaves the reader wondering what modern features are needed.
> Would it make sense to mention that "bootstrap" is used in place of
> "load" in older versions of 'launchctl'?
> 
>> +Your user-specific scheduled tasks are stored as XML-formatted `.plist`
>> +files in `~/Library/LaunchAgents/`. You can see the currently-registered
>> +tasks using the following command:
>> +
>> +-----------------------------------------------------------------------
>> +$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git
> 
> Alternately (unimportant):
> 
>     ls ~/Library/LaunchAgents/org.git-scm.git.*
> 
> although that would emit "No such file" if you don't have any
> registered, which might suggest:
> 
>     find ~/Library/LaunchAgents -name 'org.git-scm.git.*'
> 
>> +To create more advanced customizations to your background tasks, see
>> +https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
>> +for more information.
> 
> I really worry about this sort of URL becoming outdated. Would it make
> sense instead to just point the user at the man page,
> launchd.plist(5)? It's not quite the same, as it doesn't provide the
> range of examples as the URL you cite, but it should get the user
> started.

I shared similar concerns. I'll use the man page references instead.
All of the information should be a short web search away after the
user is given the right terminology.

>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1491,6 +1491,214 @@ static int maintenance_unregister(void)
>> +static int remove_plist(enum schedule_priority schedule)
>> +{
>> +       const char *frequency = get_frequency(schedule);
>> +       char *name = get_service_name(frequency);
>> +       char *filename = get_service_filename(name);
>> +       int result = bootout(filename);
>> +       free(filename);
>> +       free(name);
>> +       return result;
>> +}
>>
>> +static int remove_plists(void)
>> +{
>> +       return remove_plist(SCHEDULE_HOURLY) ||
>> +               remove_plist(SCHEDULE_DAILY) ||
>> +               remove_plist(SCHEDULE_WEEKLY);
>> +}
> 
> The new documentation you added says that the plist files will be
> deleted after they are deregistered using launchctl, but I don't see
> anything actually deleting them. Am I missing something obvious?

As mentioned below, this was a change that I made but somehow lost
while juggling multiple copies of my branch.

>> +static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
>> +{
>> +       plist = fopen(filename, "w");
>> +       if (!plist)
>> +               die(_("failed to open '%s'"), filename);
> 
> As mentioned previously, these could be replaced with a simple xfopen().
> 
> In fact, I'm having trouble seeing changes in this re-roll which you
> had planned on making, such as consolidating the repeated code in
> bootout() and bootstrap(), and ensuring that bootout() doesn't
> complain if the plist files are already missing, and so forth. Did you
> opt to not make those changes? (Which would be fine; they were minor
> suggestions.)

No, I definitely made those changes _somewhere_ but I must have
gotten confused as to which of my machines had those changes. I
guess that's part of the risk of testing across three platforms.

Thank you for noticing, and I'll be more careful from now on.

>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +       echo "#!/bin/sh\necho \$@ >>args" >print-args &&
>> +       chmod a+x print-args &&
> 
> Earlier review already mentioned write_script() and "$@". (Not
> necessarily worth a re-roll.)

I'm going to go back to all of your earlier comments to make sure
they are _actually_ applied in v3.
 
>> +       for frequency in hourly daily weekly
>> +       do
>> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
>> +               xmllint "$PLIST" >/dev/null &&
> 
> Do we really need to suppress xmllint's stdout?

It outputs the XML itself. Maybe there is a command to stop that from
happening, but nulling stdout keeps the test log clean.

Thanks,
-Stolee

Re: [PATCH v2 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Thu, Nov 12, 2020 at 8:43 AM Derrick Stolee <stolee@gmail.com> wrote:
> On 11/11/2020 3:12 AM, Eric Sunshine wrote:
> > On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >> +               xmllint "$PLIST" >/dev/null &&
> >
> > Do we really need to suppress xmllint's stdout?
>
> It outputs the XML itself. Maybe there is a command to stop that from
> happening, but nulling stdout keeps the test log clean.

xmllint's --noout option should do the trick.

[PATCH v2 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 181 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  36 +++++-
 3 files changed, 236 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 451ebac131..f4f6a4091b 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -316,6 +316,28 @@ https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy
 for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 7604064a8d..80f43a59ce 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)
 	else
 		return remove_plists();
 }
+
+#elif defined(GIT_WINDOWS_NATIVE)
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int remove_task(enum schedule_priority schedule)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+	const char *schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+
+	strvec_split(&args, schtasks);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int remove_scheduled_tasks(void)
+{
+	return remove_task(SCHEDULE_HOURLY) ||
+		remove_task(SCHEDULE_DAILY) ||
+		remove_task(SCHEDULE_WEEKLY);
+}
+
+static int schedule_task(const char *exec_path, enum schedule_priority schedule)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *xml, *schtasks;
+	char *xmlpath;
+	FILE *xmlfp;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+
+	xmlpath =  xstrfmt("%s/schedule-%s.xml",
+			   the_repository->objects->odb->path,
+			   frequency);
+	xmlfp = fopen(xmlpath, "w");
+	if (!xmlfp)
+		die(_("failed to open '%s'"), xmlpath);
+
+	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fprintf(xmlfp, xml);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml=  "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(xmlfp, xml, exec_path, exec_path, frequency);
+	fclose(xmlfp);
+
+	schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+	strvec_split(&args, schtasks);
+	strvec_pushl(&args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	unlink(xmlpath);
+	free(xmlpath);
+	free(name);
+	return result;
+}
+
+static int add_scheduled_tasks(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schedule_task(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_scheduled_tasks();
+	else
+		return remove_scheduled_tasks();
+}
+
 #else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 1c43b34a93..e7ad130cbc 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
@@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
 	test_cmp expect args
 '
 
+test_expect_success MINGW 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	for frequency in hourly daily weekly
+	do
+		printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
+			$frequency $frequency
+	done >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v2 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Git's background maintenance uses cron by default, but this is not
> available on Windows. Instead, integrate with Task Scheduler.
> [...]
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)
> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)
> +{
> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
> +                          the_repository->objects->odb->path,
> +                          frequency);

Am I reading correctly that it is writing this throwaway XML file into
the Git object directory? Would writing to a temporary directory make
more sense? (Not worth a re-roll.)

> +       xmlfp = fopen(xmlpath, "w");
> +       if (!xmlfp)
> +               die(_("failed to open '%s'"), xmlpath);

Could use xfopen() as mentioned previously.

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +test_expect_success MINGW 'start and stop Windows maintenance' '
> +       for frequency in hourly daily weekly
> +       do
> +               printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
> +                       $frequency $frequency

Nit: You lost the `|| return 1` which was present in the previous
version. True, it's very unlikely that `printf` could fail, but having
the `|| return 1` there makes it easier for the reader's eye to glide
over the code without having to worry about whether it is handling
error conditions correctly, thus reduces cognitive load.

> +       done >expect &&

Rather than a loop, you could just use:

    printf "/create ... (%s) /f /xml ...schedule-%s.xml\n" \
        hourly hourly daily daily weekly weekly >expect &&

though it's subjective as to whether that is an improvement.

Re: [PATCH v2 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/11/2020 3:59 AM, Eric Sunshine wrote:
> On Wed, Nov 4, 2020 at 3:06 PM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> Git's background maintenance uses cron by default, but this is not
>> available on Windows. Instead, integrate with Task Scheduler.
>> [...]
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1698,6 +1698,187 @@ static int platform_update_schedule(int run_maintenance, int fd)
>> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)
>> +{
>> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
>> +                          the_repository->objects->odb->path,
>> +                          frequency);
> 
> Am I reading correctly that it is writing this throwaway XML file into
> the Git object directory? Would writing to a temporary directory make
> more sense? (Not worth a re-roll.)

A temp directory is a good idea.

>> +       xmlfp = fopen(xmlpath, "w");
>> +       if (!xmlfp)
>> +               die(_("failed to open '%s'"), xmlpath);
> 
> Could use xfopen() as mentioned previously.
> 
>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -442,6 +442,36 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +test_expect_success MINGW 'start and stop Windows maintenance' '
>> +       for frequency in hourly daily weekly
>> +       do
>> +               printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
>> +                       $frequency $frequency
> 
> Nit: You lost the `|| return 1` which was present in the previous
> version. True, it's very unlikely that `printf` could fail, but having
> the `|| return 1` there makes it easier for the reader's eye to glide
> over the code without having to worry about whether it is handling
> error conditions correctly, thus reduces cognitive load.
>
>> +       done >expect &&
> 
> Rather than a loop, you could just use:
> 
>     printf "/create ... (%s) /f /xml ...schedule-%s.xml\n" \
>         hourly hourly daily daily weekly weekly >expect &&
> 
> though it's subjective as to whether that is an improvement.

It's sufficient, and avoids issues with deep tabbing and
chaining "|| return 1"".

Thanks,
-Stolee

[PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.

Updates in V3
=============

 * This actually includes the feedback responses I had intended for v2.
   Sorry about that!
   
   
 * One major change is the use of a 'struct child_process' instead of just
   run_command_v_opt() so we can suppress error messages from the schedule
   helpers. We will rely on exit code and present our own error messages, as
   necessary.
   
   
 * Some doc and test fixes.
   
   

Updates in V2
=============

 * This is a faster turnaround for a v2 than I would normally like, but Eric
   inspired extra documentation about how to customize background schedules.
   
   
 * New extensions to git-maintenance.txt include guidelines for inspecting
   what git maintenance start does and how to customize beyond that. This
   includes a new PATCH 2 that includes documentation for 'cron' on
   non-macOS non-Windows systems.
   
   
 * Several improvements, especially in the tests, are included.
   
   
 * While testing manually, I noticed that somehow I had incorrectly had an
   opening <dict> tag instead of a closing </dict> tag in the hourly format
   on macOS. I found that the xmllint tool can verify the XML format of a
   file, which catches the bug. This seems like a good approach since the
   test is macOS-only. Does anyone have concerns about adding this
   dependency?
   
   

Thanks, -Stolee

cc: jrnieder@gmail.com [jrnieder@gmail.com], jonathantanmy@google.com
[jonathantanmy@google.com], sluongng@gmail.com [sluongng@gmail.com]cc:
Derrick Stolee stolee@gmail.com [stolee@gmail.com]cc: Đoàn Trần Công Danh 
congdanhqx@gmail.com [congdanhqx@gmail.com]cc: Martin Ågren 
martin.agren@gmail.com [martin.agren@gmail.com]cc: Eric Sunshine 
sunshine@sunshineco.com [sunshine@sunshineco.com]cc: Derrick Stolee 
stolee@gmail.com [stolee@gmail.com]

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 116 +++++++++
 builtin/gc.c                      | 417 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            |  75 +++++-
 t/test-lib.sh                     |   4 +
 4 files changed, 592 insertions(+), 20 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v2:

 1:  d35f1aa162 = 1:  d35f1aa162 maintenance: extract platform-specific scheduling
 2:  709a173720 ! 2:  0dfe53092e maintenance: include 'cron' details in docs
     @@ Commit message
          baseline can provide a way forward for users who have never worked with
          cron schedules.
      
     +    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +---------------------------------------
      +
      +The standard mechanism for scheduling background tasks on POSIX systems
     -+is `cron`. This tool executes commands based on a given schedule. The
     ++is cron(8). This tool executes commands based on a given schedule. The
      +current list of user-scheduled tasks can be found by running `crontab -l`.
      +The schedule written by `git maintenance start` is similar to this:
      +
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +Any modifications within this region will be completely deleted by
      +`git maintenance stop` or overwritten by `git maintenance start`.
      +
     -+The `<path>` string is loaded to specifically use the location for the
     -+`git` executable used in the `git maintenance start` command. This allows
     -+for multiple versions to be compatible. However, if the same user runs
     -+`git maintenance start` with multiple Git executables, then only the
     -+latest executable will be used.
     ++The `crontab` entry specifies the full path of the `git` executable to
     ++ensure that the executed `git` command is the same one with which
     ++`git maintenance start` was issued independent of `PATH`. If the same user
     ++runs `git maintenance start` with multiple Git executables, then only the
     ++latest executable is used.
      +
      +These commands use `git for-each-repo --config=maintenance.repo` to run
      +`git maintenance run --schedule=<frequency>` on each repository listed in
      +the multi-valued `maintenance.repo` config option. These are typically
     -+loaded from the user-specific global config located at `~/.gitconfig`.
     -+The `git maintenance` process then determines which maintenance tasks
     -+are configured to run on each repository with each `<frequency>` using
     -+the `maintenance.<task>.schedule` config options. These values are loaded
     -+from the global or repository config values.
     ++loaded from the user-specific global config. The `git maintenance` process
     ++then determines which maintenance tasks are configured to run on each
     ++repository with each `<frequency>` using the `maintenance.<task>.schedule`
     ++config options. These values are loaded from the global or repository
     ++config values.
      +
      +If the config values are insufficient to achieve your desired background
      +maintenance schedule, then you can create your own schedule. If you run
      +`crontab -e`, then an editor will load with your user-specific `cron`
      +schedule. In that editor, you can add your own schedule lines. You could
      +start by adapting the default schedule listed earlier, or you could read
     -+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
     -+for advanced scheduling techniques. Please do use the full path and
     -+`--exec-path` techniques from the default schedule to ensure you are
     -+executing the correct binaries in your schedule.
     ++the crontab(5) documentation for advanced scheduling techniques. Please
     ++do use the full path and `--exec-path` techniques from the default
     ++schedule to ensure you are executing the correct binaries in your
     ++schedule.
      +
       
       GIT
 3:  0fafd75d10 ! 3:  1629bcfcf8 maintenance: use launchctl on macOS
     @@ Commit message
          of macOS 10.11, which was released in September 2015. Before that
          release the 'launchctl load' subcommand was recommended. The best
          source of information on this transition I have seen is available
     -    at [2].
     +    at [2]. The current design does not preclude a future version that
     +    detects the available fatures of 'launchctl' to use the older
     +    commands. However, it is best to rely on the newest version since
     +    Apple might completely remove the deprecated version on short
     +    notice.
      
          [2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/
      
     @@ Commit message
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     -@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
     - executing the correct binaries in your schedule.
     +@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your
     + schedule.
       
       
      +BACKGROUND MAINTENANCE ON MACOS SYSTEMS
      +---------------------------------------
      +
      +While macOS technically supports `cron`, using `crontab -e` requires
     -+elevated privileges and the executed process do not have a full user
     ++elevated privileges and the executed process does not have a full user
      +context. Without a full user context, Git and its credential helpers
      +cannot access stored credentials, so some maintenance tasks are not
      +functional.
      +
      +Instead, `git maintenance start` interacts with the `launchctl` tool,
     -+which is the recommended way to
     -+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html[schedule timed jobs in macOS].
     -+
     -+Scheduling maintenance through `git maintenance (start|stop)` requires
     -+some `launchctl` features available only in macOS 10.11 or later.
     ++which is the recommended way to schedule timed jobs in macOS. Scheduling
     ++maintenance through `git maintenance (start|stop)` requires some
     ++`launchctl` features available only in macOS 10.11 or later.
      +
      +Your user-specific scheduled tasks are stored as XML-formatted `.plist`
      +files in `~/Library/LaunchAgents/`. You can see the currently-registered
      +tasks using the following command:
      +
      +-----------------------------------------------------------------------
     -+$ ls ~/Library/LaunchAgents/ | grep org.git-scm.git
     ++$ ls ~/Library/LaunchAgents/org.git-scm.git*
      +org.git-scm.git.daily.plist
      +org.git-scm.git.hourly.plist
      +org.git-scm.git.weekly.plist
     @@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do
      +and delete the `.plist` files.
      +
      +To create more advanced customizations to your background tasks, see
     -+https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html#//apple_ref/doc/uid/TP40001762-104142[the `launchctl` documentation]
     -+for more information.
     ++launchctl.plist(5) for more information.
      +
      +
       GIT
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +	return strbuf_detach(&output, NULL);
      +}
      +
     -+static int bootout(const char *filename)
     ++static int boot_plist(int enable, const char *filename)
      +{
      +	int result;
     -+	struct strvec args = STRVEC_INIT;
     ++	struct child_process child = CHILD_PROCESS_INIT;
      +	char *uid = get_uid();
      +	const char *launchctl = getenv("GIT_TEST_CRONTAB");
      +	if (!launchctl)
      +		launchctl = "/bin/launchctl";
      +
     -+	strvec_split(&args, launchctl);
     -+	strvec_push(&args, "bootout");
     -+	strvec_pushf(&args, "gui/%s", uid);
     -+	strvec_push(&args, filename);
     ++	strvec_split(&child.args, launchctl);
      +
     -+	result = run_command_v_opt(args.v, 0);
     ++	if (enable)
     ++		strvec_push(&child.args, "bootstrap");
     ++	else
     ++		strvec_push(&child.args, "bootout");
     ++	strvec_pushf(&child.args, "gui/%s", uid);
     ++	strvec_push(&child.args, filename);
      +
     -+	strvec_clear(&args);
     -+	free(uid);
     -+	return result;
     -+}
     ++	child.no_stderr = 1;
     ++	child.no_stdout = 1;
      +
     -+static int bootstrap(const char *filename)
     -+{
     -+	int result;
     -+	struct strvec args = STRVEC_INIT;
     -+	char *uid = get_uid();
     -+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
     -+	if (!launchctl)
     -+		launchctl = "/bin/launchctl";
     ++	if (start_command(&child))
     ++		die(_("failed to start launchctl"));
      +
     -+	strvec_split(&args, launchctl);
     -+	strvec_push(&args, "bootstrap");
     -+	strvec_pushf(&args, "gui/%s", uid);
     -+	strvec_push(&args, filename);
     ++	result = finish_command(&child);
      +
     -+	result = run_command_v_opt(args.v, 0);
     -+
     -+	strvec_clear(&args);
      +	free(uid);
      +	return result;
      +}
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +	const char *frequency = get_frequency(schedule);
      +	char *name = get_service_name(frequency);
      +	char *filename = get_service_filename(name);
     -+	int result = bootout(filename);
     ++	int result = boot_plist(0, filename);
     ++	unlink(filename);
      +	free(filename);
      +	free(name);
      +	return result;
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +
      +	if (safe_create_leading_directories(filename))
      +		die(_("failed to create directories for '%s'"), filename);
     -+	plist = fopen(filename, "w");
     -+
     -+	if (!plist)
     -+		die(_("failed to open '%s'"), filename);
     ++	plist = xfopen(filename, "w");
      +
      +	preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
      +		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +	fprintf(plist, "</array>\n</dict>\n</plist>\n");
      +
      +	/* bootout might fail if not already running, so ignore */
     -+	bootout(filename);
     -+	if (bootstrap(filename))
     ++	boot_plist(0, filename);
     ++	if (boot_plist(1, filename))
      +		die(_("failed to bootstrap service %s"), filename);
      +
      +	fclose(plist);
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
       '
       
      +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
     -+	echo "#!/bin/sh\necho \$@ >>args" >print-args &&
     -+	chmod a+x print-args &&
     ++	write_script print-args "#!/bin/sh\necho \$* >>args" &&
      +
      +	rm -f args &&
      +	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
      +	for frequency in hourly daily weekly
      +	do
      +		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
     -+		xmllint "$PLIST" >/dev/null &&
     ++		xmllint --noout "$PLIST" &&
      +		grep schedule=$frequency "$PLIST" &&
      +		echo "bootout gui/$UID $PLIST" >>expect &&
      +		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
      +	test_cmp expect args &&
      +
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="./print-args"  git maintenance stop &&
     ++	GIT_TEST_CRONTAB="./print-args" git maintenance stop &&
      +
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	# stop does not remove plist files, but boots them out
     -+	rm expect &&
     -+	for frequency in hourly daily weekly
     -+	do
     -+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
     -+		grep schedule=$frequency "$PLIST" &&
     -+		echo "bootout gui/$UID $PLIST" >>expect || return 1
     -+	done &&
     -+	test_cmp expect args
     ++	printf "bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
     ++		hourly daily weekly >expect &&
     ++	test_cmp expect args &&
     ++	ls "$HOME/Library/LaunchAgents" >actual &&
     ++	test_line_count = 0 actual
      +'
      +
       test_expect_success 'register preserves existing strategy' '
 4:  84eb44de31 ! 4:  ed7a61978f maintenance: use Windows scheduled tasks
     @@ Commit message
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     -@@ Documentation/git-maintenance.txt: https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSy
     - for more information.
     +@@ Documentation/git-maintenance.txt: To create more advanced customizations to your background tasks, see
     + launchctl.plist(5) for more information.
       
       
      +BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +static int schedule_task(const char *exec_path, enum schedule_priority schedule)
      +{
      +	int result;
     -+	struct strvec args = STRVEC_INIT;
     ++	struct child_process child = CHILD_PROCESS_INIT;
      +	const char *xml, *schtasks;
     -+	char *xmlpath;
     ++	char *xmlpath, *tempDir;
      +	FILE *xmlfp;
      +	const char *frequency = get_frequency(schedule);
      +	char *name = get_task_name(frequency);
      +
     -+	xmlpath =  xstrfmt("%s/schedule-%s.xml",
     -+			   the_repository->objects->odb->path,
     -+			   frequency);
     -+	xmlfp = fopen(xmlpath, "w");
     -+	if (!xmlfp)
     -+		die(_("failed to open '%s'"), xmlpath);
     ++	tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
     ++	xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
     ++	safe_create_leading_directories(xmlpath);
     ++	xmlfp = xfopen(xmlpath, "w");
      +
      +	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
      +	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +	schtasks = getenv("GIT_TEST_CRONTAB");
      +	if (!schtasks)
      +		schtasks = "schtasks";
     -+	strvec_split(&args, schtasks);
     -+	strvec_pushl(&args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
     ++	strvec_split(&child.args, schtasks);
     ++	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
      +
     -+	result = run_command_v_opt(args.v, 0);
     ++	child.no_stdout = 1;
     ++	child.no_stderr = 1;
     ++
     ++	if (start_command(&child))
     ++		die(_("failed to start schtasks"));
     ++	result = finish_command(&child);
      +
     -+	strvec_clear(&args);
      +	unlink(xmlpath);
     ++	rmdir(tempDir);
      +	free(xmlpath);
      +	free(name);
      +	return result;
     @@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'stop from existi
       	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
       	grep "Important information!" cron.txt
      @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
     - 	test_cmp expect args
     + 	test_line_count = 0 actual
       '
       
      +test_expect_success MINGW 'start and stop Windows maintenance' '
     @@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop ma
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	for frequency in hourly daily weekly
     -+	do
     -+		printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
     -+			$frequency $frequency
     -+	done >expect &&
     ++	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\n" \
     ++		hourly hourly daily daily weekly weekly >expect &&
      +	test_cmp expect args &&
      +
      +	rm -f args &&

-- 
gitgitgadget

[PATCH v3 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c | 38 +++++++++++++++++++++-----------------
 1 file changed, 21 insertions(+), 17 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a1..c1f7d9bdc2 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,7 +1494,7 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int platform_update_schedule(int run_maintenance, int fd)
 {
 	int result = 0;
 	int in_old_region = 0;
@@ -1503,11 +1503,6 @@ static int update_background_schedule(int run_maintenance)
 	FILE *cron_list, *cron_in;
 	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
-
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
 
 	crontab_name = getenv("GIT_TEST_CRONTAB");
 	if (!crontab_name)
@@ -1516,12 +1511,11 @@ static int update_background_schedule(int run_maintenance)
 	strvec_split(&crontab_list.args, crontab_name);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
 	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 	}
 
 	/* Ignore exit code, as an empty crontab will return error. */
@@ -1531,7 +1525,7 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
 	strvec_split(&crontab_edit.args, crontab_name);
@@ -1539,8 +1533,7 @@ static int update_background_schedule(int run_maintenance)
 	crontab_edit.git_cmd = 0;
 
 	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 	}
 
 	cron_in = fdopen(crontab_edit.in, "w");
@@ -1586,13 +1579,24 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
-	}
-	fclose(cron_list);
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static int update_background_schedule(int run_maintenance)
+{
+	int result;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	result = platform_update_schedule(run_maintenance, lk.tempfile->fd);
 
-cleanup:
 	rollback_lock_file(&lk);
 	return result;
 }
-- 
gitgitgadget

[PATCH v3 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc..1aa1112418 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is cron(8). This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `crontab` entry specifies the full path of the `git` executable to
+ensure that the executed `git` command is the same one with which
+`git maintenance start` was issued independent of `PATH`. If the same user
+runs `git maintenance start` with multiple Git executables, then only the
+latest executable is used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config. The `git maintenance` process
+then determines which maintenance tasks are configured to run on each
+repository with each `<frequency>` using the `maintenance.<task>.schedule`
+config options. These values are loaded from the global or repository
+config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+the crontab(5) documentation for advanced scheduling techniques. Please
+do use the full path and `--exec-path` techniques from the default
+schedule to ensure you are executing the correct binaries in your
+schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

[PATCH v3 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2]. The current design does not preclude a future version that
detects the available fatures of 'launchctl' to use the older
commands. However, it is best to rely on the newest version since
Apple might completely remove the deprecated version on short
notice.

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_CRONTAB.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format, so call it from the macOS-specific tests to ensure
the .plist files are well-formatted.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  40 ++++++
 builtin/gc.c                      | 195 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  48 +++++++-
 t/test-lib.sh                     |   4 +
 4 files changed, 284 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1aa1112418..5f8f63f098 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process does not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to schedule timed jobs in macOS. Scheduling
+maintenance through `git maintenance (start|stop)` requires some
+`launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/org.git-scm.git*
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+launchctl.plist(5) for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index c1f7d9bdc2..da2c892f68 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+#if defined(__APPLE__)
+
+static char *get_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *get_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_uid(void)
+{
+	struct strbuf output = STRBUF_INIT;
+	struct child_process id = CHILD_PROCESS_INIT;
+
+	strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL);
+	if (capture_command(&id, &output, 0))
+		die(_("failed to discover user id"));
+
+	strbuf_trim_trailing_newline(&output);
+	return strbuf_detach(&output, NULL);
+}
+
+static int boot_plist(int enable, const char *filename)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	char *uid = get_uid();
+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
+	if (!launchctl)
+		launchctl = "/bin/launchctl";
+
+	strvec_split(&child.args, launchctl);
+
+	if (enable)
+		strvec_push(&child.args, "bootstrap");
+	else
+		strvec_push(&child.args, "bootout");
+	strvec_pushf(&child.args, "gui/%s", uid);
+	strvec_push(&child.args, filename);
+
+	child.no_stderr = 1;
+	child.no_stdout = 1;
+
+	if (start_command(&child))
+		die(_("failed to start launchctl"));
+
+	result = finish_command(&child);
+
+	free(uid);
+	return result;
+}
+
+static int remove_plist(enum schedule_priority schedule)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+	int result = boot_plist(0, filename);
+	unlink(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int remove_plists(void)
+{
+	return remove_plist(SCHEDULE_HOURLY) ||
+		remove_plist(SCHEDULE_DAILY) ||
+		remove_plist(SCHEDULE_WEEKLY);
+}
+
+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_service_name(frequency);
+	char *filename = get_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = xfopen(filename, "w");
+
+	preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+
+	/* bootout might fail if not already running, so ignore */
+	boot_plist(0, filename);
+	if (boot_plist(1, filename))
+		die(_("failed to bootstrap service %s"), filename);
+
+	fclose(plist);
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int add_plists(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_plist(exec_path, SCHEDULE_HOURLY) ||
+		schedule_plist(exec_path, SCHEDULE_DAILY) ||
+		schedule_plist(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_plists();
+	else
+		return remove_plists();
+}
+#else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1585,6 +1779,7 @@ static int platform_update_schedule(int run_maintenance, int fd)
 		fclose(cron_list);
 	return result;
 }
+#endif
 
 static int update_background_schedule(int run_maintenance)
 {
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1..29d340a828 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
+	write_script print-args "#!/bin/sh\necho \$* >>args" &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	# ~/Library/LaunchAgents
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		xmllint --noout "$PLIST" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$UID $PLIST" >>expect &&
+		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args &&
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	test_line_count = 0 actual
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..620ffbf3af 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1703,6 +1703,10 @@ test_lazy_prereq REBASE_P '
 	test -z "$GIT_TEST_SKIP_REBASE_P"
 '
 
+test_lazy_prereq MACOS_MAINTENANCE '
+	launchctl list
+'
+
 # Ensure that no test accidentally triggers a Git command
 # that runs 'crontab', affecting a user's cron schedule.
 # Tests that verify the cron integration must set this locally
-- 
gitgitgadget

Re: [PATCH v3 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> [...]
> The solution is to switch from cron to the Apple-recommended [1]
> 'launchd' tool.
> [...]
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)
> +static int boot_plist(int enable, const char *filename)
> +{
> +       struct child_process child = CHILD_PROCESS_INIT;
> +       child.no_stderr = 1;
> +       child.no_stdout = 1;
> +       if (start_command(&child))
> +               die(_("failed to start launchctl"));

Not necessarily worth a re-roll -- in fact, it could be done atop this
series to avoid holding this series up -- but this too-succinct error
reporting won't help users diagnose the failure. An alternative would
be to capture stdout and stderr and only print them if the command
fails. Perhaps something like this:

    struct strbuf out = STRBUF_INIT;
    struct strbuf err = STRBUF_INIT;
    ...
    if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {
        if (out.len && err.len)
            strbuf_addstr(&out, "; ");
        strbuf_addbuf(&out, &err);
        die(_("launchctl failed: %s"), out.buf);
    }

By the way, won't this die() be a problem when schedule_plist() calls
boot_plist() to remove the old scheduled tasks before calling it again
to register the new ones? If the old ones don't exist, then it will
die() unnecessarily and never register the new ones. Or am I
misunderstanding? (I'm guessing that I must be misunderstanding since
the test script presumably passes.)

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '
> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +       write_script print-args "#!/bin/sh\necho \$* >>args" &&

write_script() takes the script body as stdin, not as an argument, and
you don't need to specify /bin/sh. What you have here works by
accident only because write_script() takes an optional second argument
specifying the shell to use in place of the default /bin/sh.
Nevertheless, it should really be written:

    write_script print-args <<-\EOF
    echo $*
    EOF

Patch [4/4] uses write_script() correctly.

> +       rm -f args &&
> +       GIT_TEST_CRONTAB="./print-args" git maintenance start &&
> +
> +       # start registers the repo
> +       git config --get --global maintenance.repo "$(pwd)" &&
> +
> +       # ~/Library/LaunchAgents
> +       ls "$HOME/Library/LaunchAgents" >actual &&

Not sure what the comment above the `ls` is meant to be conveying.
Could be dropped but not itself worth a re-roll.

> +       cat >expect <<-\EOF &&
> +       org.git-scm.git.daily.plist
> +       org.git-scm.git.hourly.plist
> +       org.git-scm.git.weekly.plist
> +       EOF
> +       test_cmp expect actual &&
> +
> +       rm expect &&
> +       for frequency in hourly daily weekly
> +       do
> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
> +               xmllint --noout "$PLIST" &&
> +               grep schedule=$frequency "$PLIST" &&
> +               echo "bootout gui/$UID $PLIST" >>expect &&
> +               echo "bootstrap gui/$UID $PLIST" >>expect || return 1
> +       done &&

This is still relying upon $UID picked up from the users environment
(as far as I can tell), which seems fragile. As mentioned in my first
review, it probably would be more robust to compute UID manually the
same way git-maintenance itself does.

> +       test_cmp expect args &&
> +
> +       rm -f args &&
> +       GIT_TEST_CRONTAB="./print-args" git maintenance stop &&

Minor: No need for the quotes around ./print-args (though they don't
hurt either, and certainly not worth re-rolling just to drop them, and
it's subjective so don't drop them just for my sake).

> +       # stop does not unregister the repo
> +       git config --get --global maintenance.repo "$(pwd)" &&
> +
> +       printf "bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
> +               hourly daily weekly >expect &&
> +       test_cmp expect args &&
> +       ls "$HOME/Library/LaunchAgents" >actual &&
> +       test_line_count = 0 actual
> +'

Re: [PATCH v3 3/4] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/13/2020 3:19 PM, Eric Sunshine wrote:
> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> [...]
>> The solution is to switch from cron to the Apple-recommended [1]
>> 'launchd' tool.
>> [...]
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1491,6 +1491,200 @@ static int maintenance_unregister(void)
>> +static int boot_plist(int enable, const char *filename)
>> +{
>> +       struct child_process child = CHILD_PROCESS_INIT;
>> +       child.no_stderr = 1;
>> +       child.no_stdout = 1;
>> +       if (start_command(&child))
>> +               die(_("failed to start launchctl"));
> 
> Not necessarily worth a re-roll -- in fact, it could be done atop this
> series to avoid holding this series up -- but this too-succinct error
> reporting won't help users diagnose the failure. An alternative would
> be to capture stdout and stderr and only print them if the command
> fails. Perhaps something like this:
> 
>     struct strbuf out = STRBUF_INIT;
>     struct strbuf err = STRBUF_INIT;
>     ...
>     if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {
>         if (out.len && err.len)
>             strbuf_addstr(&out, "; ");
>         strbuf_addbuf(&out, &err);
>         die(_("launchctl failed: %s"), out.buf);
>     }

We would also want to pass a "die_on_failure" into the method, since
in the 'git maintenance start' case we don't want to report a failure
when 'launchctl bootout' fails before we call 'launchctl bootstrap'.

> By the way, won't this die() be a problem when schedule_plist() calls
> boot_plist() to remove the old scheduled tasks before calling it again
> to register the new ones? If the old ones don't exist, then it will
> die() unnecessarily and never register the new ones. Or am I
> misunderstanding? (I'm guessing that I must be misunderstanding since
> the test script presumably passes.)

This die() is only if the process cannot _start_, for example due to
launchctl not existing on $PATH. The result from finish_command()
would be non-zero when we bootout a plist that doesn't exist.

>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -389,12 +389,54 @@ test_expect_success 'stop from existing schedule' '
>> +test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +       write_script print-args "#!/bin/sh\necho \$* >>args" &&
> 
> write_script() takes the script body as stdin, not as an argument, and
> you don't need to specify /bin/sh. What you have here works by
> accident only because write_script() takes an optional second argument
> specifying the shell to use in place of the default /bin/sh.
> Nevertheless, it should really be written:
> 
>     write_script print-args <<-\EOF
>     echo $*
>     EOF
> 
> Patch [4/4] uses write_script() correctly.

Ah. Sorry for misunderstanding. That explains why it works this way
on macOS but it did _not_ work that way on Windows.

>> +       rm -f args &&
>> +       GIT_TEST_CRONTAB="./print-args" git maintenance start &&
>> +
>> +       # start registers the repo
>> +       git config --get --global maintenance.repo "$(pwd)" &&
>> +
>> +       # ~/Library/LaunchAgents
>> +       ls "$HOME/Library/LaunchAgents" >actual &&
> 
> Not sure what the comment above the `ls` is meant to be conveying.
> Could be dropped but not itself worth a re-roll.
> 
>> +       cat >expect <<-\EOF &&
>> +       org.git-scm.git.daily.plist
>> +       org.git-scm.git.hourly.plist
>> +       org.git-scm.git.weekly.plist
>> +       EOF
>> +       test_cmp expect actual &&
>> +
>> +       rm expect &&
>> +       for frequency in hourly daily weekly
>> +       do
>> +               PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
>> +               xmllint --noout "$PLIST" &&
>> +               grep schedule=$frequency "$PLIST" &&
>> +               echo "bootout gui/$UID $PLIST" >>expect &&
>> +               echo "bootstrap gui/$UID $PLIST" >>expect || return 1
>> +       done &&
> 
> This is still relying upon $UID picked up from the users environment
> (as far as I can tell), which seems fragile. As mentioned in my first
> review, it probably would be more robust to compute UID manually the
> same way git-maintenance itself does.

Sorry, I missed this comment from v1 when reapplying the changes for v3.

>> +       test_cmp expect args &&
>> +
>> +       rm -f args &&
>> +       GIT_TEST_CRONTAB="./print-args" git maintenance stop &&
> 
> Minor: No need for the quotes around ./print-args (though they don't
> hurt either, and certainly not worth re-rolling just to drop them, and
> it's subjective so don't drop them just for my sake).

Thank you for your continued attention and patience.
-Stolee

Re: [PATCH v3 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Fri, Nov 13, 2020 at 3:42 PM Derrick Stolee <stolee@gmail.com> wrote:
> On 11/13/2020 3:19 PM, Eric Sunshine wrote:
> >     if (pipe_command(child, NULL, 0, &out, 0, &err, 0) {
> >         if (out.len && err.len)
> >             strbuf_addstr(&out, "; ");
> >         strbuf_addbuf(&out, &err);
> >         die(_("launchctl failed: %s"), out.buf);
> >     }
>
> We would also want to pass a "die_on_failure" into the method, since
> in the 'git maintenance start' case we don't want to report a failure
> when 'launchctl bootout' fails before we call 'launchctl bootstrap'.

Right. I started writing that we'd also need a `die_one_failure` flag
but deleted the comment since I decided to wait until I got an
answer...

> > By the way, won't this die() be a problem when schedule_plist() calls
> > boot_plist() to remove the old scheduled tasks before calling it again
> > to register the new ones? If the old ones don't exist, then it will
> > die() unnecessarily and never register the new ones. Or am I
> > misunderstanding? (I'm guessing that I must be misunderstanding since
> > the test script presumably passes.)
>
> This die() is only if the process cannot _start_, for example due to
> launchctl not existing on $PATH. The result from finish_command()
> would be non-zero when we bootout a plist that doesn't exist.

... to this question.

Another thought I had was simply checking for the presence of the file
and skipping `bootout` altogether if it doesn't exist. That would, I
think, obviate the need for mucking with stdout/stderr oppression.

> > write_script() takes the script body as stdin, not as an argument, and
> > you don't need to specify /bin/sh. What you have here works by
> > accident only because write_script() takes an optional second argument
> > specifying the shell to use in place of the default /bin/sh.
> > Nevertheless, it should really be written:
> >
> >     write_script print-args <<-\EOF
> >     echo $*
> >     EOF
> >
> > Patch [4/4] uses write_script() correctly.
>
> Ah. Sorry for misunderstanding. That explains why it works this way
> on macOS but it did _not_ work that way on Windows.

Sorry on my part too. I missed the `args` redirect in my example. It should be:

    write_script print-args <<-\EOF
    echo $* >args
    EOF

Re: [PATCH v3 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Fri, Nov 13, 2020 at 3:53 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> Another thought I had was simply checking for the presence of the file
> and skipping `bootout` altogether if it doesn't exist. That would, I
> think, obviate the need for mucking with stdout/stderr oppression.

Erm, s/oppression/suppression/.

[PATCH v3 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 184 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  33 +++++-
 3 files changed, 236 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 5f8f63f098..6970f2b898 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see
 launchctl.plist(5) for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index da2c892f68..76a3afa20a 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)
 	else
 		return remove_plists();
 }
+
+#elif defined(GIT_WINDOWS_NATIVE)
+
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *get_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int remove_task(enum schedule_priority schedule)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+	const char *schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+
+	strvec_split(&args, schtasks);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int remove_scheduled_tasks(void)
+{
+	return remove_task(SCHEDULE_HOURLY) ||
+		remove_task(SCHEDULE_DAILY) ||
+		remove_task(SCHEDULE_WEEKLY);
+}
+
+static int schedule_task(const char *exec_path, enum schedule_priority schedule)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *xml, *schtasks;
+	char *xmlpath, *tempDir;
+	FILE *xmlfp;
+	const char *frequency = get_frequency(schedule);
+	char *name = get_task_name(frequency);
+
+	tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
+	xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
+	safe_create_leading_directories(xmlpath);
+	xmlfp = xfopen(xmlpath, "w");
+
+	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fprintf(xmlfp, xml);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(xmlfp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml=  "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(xmlfp, xml, exec_path, exec_path, frequency);
+	fclose(xmlfp);
+
+	schtasks = getenv("GIT_TEST_CRONTAB");
+	if (!schtasks)
+		schtasks = "schtasks";
+	strvec_split(&child.args, schtasks);
+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
+
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+
+	if (start_command(&child))
+		die(_("failed to start schtasks"));
+	result = finish_command(&child);
+
+	unlink(xmlpath);
+	rmdir(tempDir);
+	free(xmlpath);
+	free(name);
+	return result;
+}
+
+static int add_scheduled_tasks(void)
+{
+	const char *exec_path = git_exec_path();
+
+	return schedule_task(exec_path, SCHEDULE_HOURLY) ||
+		schedule_task(exec_path, SCHEDULE_DAILY) ||
+		schedule_task(exec_path, SCHEDULE_WEEKLY);
+}
+
+static int platform_update_schedule(int run_maintenance, int fd)
+{
+	if (run_maintenance)
+		return add_scheduled_tasks();
+	else
+		return remove_scheduled_tasks();
+}
+
 #else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 29d340a828..0dc2479117 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -367,7 +367,7 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
@@ -378,7 +378,7 @@ test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
@@ -389,7 +389,7 @@ test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
 	test_must_be_empty cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
@@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
 	test_line_count = 0 actual
 '
 
+test_expect_success MINGW 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\n" \
+		hourly hourly daily daily weekly weekly >expect &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Git's background maintenance uses cron by default, but this is not
> available on Windows. Instead, integrate with Task Scheduler.
> [...]
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)
> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)
> +{
> +       char *xmlpath, *tempDir;
> +       tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
> +       xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);

When I wondered aloud in my previous review whether writing these
throwaway files to a temporary directory would make sense, I was
thinking more along the lines of /tmp or $TEMP. More specifically, we
have xmkstemp() in wrapper.c which is good for this sort of thing (or
one of the other temporary-file-making functions in there). We also
have a more full-featured temporary-file API in tempfile.h which would
ensure that these throwaway files actually get thrown away when the
command finishes.

This is not necessarily worth a re-roll.

> +       if (start_command(&child))
> +               die(_("failed to start schtasks"));
> +       result = finish_command(&child);
> +
> +       unlink(xmlpath);
> +       rmdir(tempDir);

Neither xmlpath and tempDir get cleaned up from the filesystem if the
preceding die() is triggered (which may or may not make sense --
perhaps you want to keep them around if it helps with the diagnosis of
the failure). The functions in tempfile.h would ensure the temporary
file is cleaned up even if the program die()s, or you could manually
remove the temporary file before die()ing.

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
> +test_expect_success MINGW 'start and stop Windows maintenance' '
> +       write_script print-args <<-\EOF &&
> +       echo $* >>args
> +       EOF

Using `>>` here makes it harder to reason about the test than using
`>` would, especially since `>>` seems to be unnecessary in this case.

> +       rm -f args &&
> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&

Is it a requirement on Windows to mention /bin/sh here? Specifically,
I'm wondering why a simple ./print-args doesn't work. (It's especially
unclear since write_script() is used heavily in the test suite and it
seems to work well enough on Windows without specifying /bin/sh.)

Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/13/2020 3:44 PM, Eric Sunshine wrote:
> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> Git's background maintenance uses cron by default, but this is not
>> available on Windows. Instead, integrate with Task Scheduler.
>> [...]
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1684,6 +1684,190 @@ static int platform_update_schedule(int run_maintenance, int fd)
>> +static int schedule_task(const char *exec_path, enum schedule_priority schedule)
>> +{
>> +       char *xmlpath, *tempDir;
>> +       tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
>> +       xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
> 
> When I wondered aloud in my previous review whether writing these
> throwaway files to a temporary directory would make sense, I was
> thinking more along the lines of /tmp or $TEMP. More specifically, we
> have xmkstemp() in wrapper.c which is good for this sort of thing (or
> one of the other temporary-file-making functions in there). We also
> have a more full-featured temporary-file API in tempfile.h which would
> ensure that these throwaway files actually get thrown away when the
> command finishes.
> 
> This is not necessarily worth a re-roll.
> 
>> +       if (start_command(&child))
>> +               die(_("failed to start schtasks"));
>> +       result = finish_command(&child);
>> +
>> +       unlink(xmlpath);
>> +       rmdir(tempDir);
> 
> Neither xmlpath and tempDir get cleaned up from the filesystem if the
> preceding die() is triggered (which may or may not make sense --
> perhaps you want to keep them around if it helps with the diagnosis of
> the failure). The functions in tempfile.h would ensure the temporary
> file is cleaned up even if the program die()s, or you could manually
> remove the temporary file before die()ing.

While I do like to have access to the data when trying to resolve
an issue, it's probably better to use the tempfile library.

>> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
>> @@ -437,6 +437,33 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
>> +test_expect_success MINGW 'start and stop Windows maintenance' '
>> +       write_script print-args <<-\EOF &&
>> +       echo $* >>args
>> +       EOF
> 
> Using `>>` here makes it harder to reason about the test than using
> `>` would, especially since `>>` seems to be unnecessary in this case.

Since we execute the GIT_TEST_CRONTAB executable multiple times, we
need to use >> to log all three instances (and their order). Using ">args"
would only capture the final call for the weekly schedule.

On macOS, there are as many as six calls (three bootouts, three bootstraps).

>> +       rm -f args &&
>> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
> 
> Is it a requirement on Windows to mention /bin/sh here? Specifically,
> I'm wondering why a simple ./print-args doesn't work. (It's especially
> unclear since write_script() is used heavily in the test suite and it
> seems to work well enough on Windows without specifying /bin/sh.)

I landed on this after trying several attempts to get this to work,
including "$(pwd)/print-args" and I'm not sure why it doesn't work
in the Windows case. It is something to do with how I am executing
the subcommand from within Git. I'm pretty sure this idea of "mocking"
an executable through Git is relatively new, or at least rare

Thanks,
-Stolee

Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Fri, Nov 13, 2020 at 4:32 PM Derrick Stolee <stolee@gmail.com> wrote:
> On 11/13/2020 3:44 PM, Eric Sunshine wrote:
> > On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >> +test_expect_success MINGW 'start and stop Windows maintenance' '
> >> +       write_script print-args <<-\EOF &&
> >> +       echo $* >>args
> >> +       EOF
> >
> > Using `>>` here makes it harder to reason about the test than using
> > `>` would, especially since `>>` seems to be unnecessary in this case.
>
> Since we execute the GIT_TEST_CRONTAB executable multiple times, we
> need to use >> to log all three instances (and their order). Using ">args"
> would only capture the final call for the weekly schedule.
>
> On macOS, there are as many as six calls (three bootouts, three bootstraps).

Makes sense. Thanks.

> >> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
> >
> > Is it a requirement on Windows to mention /bin/sh here? Specifically,
> > I'm wondering why a simple ./print-args doesn't work. (It's especially
> > unclear since write_script() is used heavily in the test suite and it
> > seems to work well enough on Windows without specifying /bin/sh.)
>
> I landed on this after trying several attempts to get this to work,
> including "$(pwd)/print-args" and I'm not sure why it doesn't work
> in the Windows case. It is something to do with how I am executing
> the subcommand from within Git. I'm pretty sure this idea of "mocking"
> an executable through Git is relatively new, or at least rare

Just for clarification... You mentioned in response to my [3/4] review
that your accidentally-working write_script() only worked as expected
on Mac but not on Windows. When you arrived at this solution of
GIT_TEST_CRONTAB="/bin/sh ..." here, was that before or after you
fixed write_script() to take the script body from stdin?

Re: [PATCH v3 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/13/2020 4:40 PM, Eric Sunshine wrote:
> On Fri, Nov 13, 2020 at 4:32 PM Derrick Stolee <stolee@gmail.com> wrote:
>> On 11/13/2020 3:44 PM, Eric Sunshine wrote:
>>> On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
>>>> +       GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
>>>
>>> Is it a requirement on Windows to mention /bin/sh here? Specifically,
>>> I'm wondering why a simple ./print-args doesn't work. (It's especially
>>> unclear since write_script() is used heavily in the test suite and it
>>> seems to work well enough on Windows without specifying /bin/sh.)
>>
>> I landed on this after trying several attempts to get this to work,
>> including "$(pwd)/print-args" and I'm not sure why it doesn't work
>> in the Windows case. It is something to do with how I am executing
>> the subcommand from within Git. I'm pretty sure this idea of "mocking"
>> an executable through Git is relatively new, or at least rare
> 
> Just for clarification... You mentioned in response to my [3/4] review
> that your accidentally-working write_script() only worked as expected
> on Mac but not on Windows. When you arrived at this solution of
> GIT_TEST_CRONTAB="/bin/sh ..." here, was that before or after you
> fixed write_script() to take the script body from stdin?

You're right. That was necessary only for the old way that I was
creating the script. The correct way works with GIT_TEST_CRONTAB
equal to ./print-args.

-Stolee

Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Fri, Nov 13, 2020 at 9:00 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>  * This actually includes the feedback responses I had intended for v2.
>    Sorry about that!

I forgot to mention a couple things when reviewing the patches
individually, so I'll point them out here...

>      +    at [2]. The current design does not preclude a future version that
>      +    detects the available fatures of 'launchctl' to use the older

s/fatures/features/

>      -+ test_cmp expect args
>      ++ test_line_count = 0 actual

These days, we usually say:

    test_must_be_empty actual

Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Fri, Nov 13, 2020 at 03:47:15PM -0500, Eric Sunshine wrote:
> I forgot to mention a couple things when reviewing the patches
> individually, so I'll point them out here...

In v2, you added an `xmllint` check on MacOS after discovering that
gc.c was generating a malformed .plist file on that platform. That got
me thinking that it would have been nice to have caught the problem
earlier, if possible, even without having access to MacOS. Since none
of the code added to gc.c has a hard platform dependency, it should be
possible to perform all the tests on any platform rather than
restricting them to specific platforms via test prerequisites. The
patch below, which is built atop v3, does just that. It removes the
conditional compilation directives from gc.c and the prerequisites
from the test script so that all scheduler-specific code in gc.c is
tested on all platform.

The changes made by the patch are intended to be folded into each of
your patches where appropriate (rather than existing atop your series,
which, though possible, would be ugly). If you're interested in
incorporating any of these improvements into v4, you can have my
"Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>" in addition
to the Helped-by: you already added.

A few more notes...

In addition to making it possible to test all platform-specific
schedulers on each platform, I also made a few other
changes/enhancements:

* simplified UID retrieval and eliminated platform-specific
  dependencies (though this may need some additional tweaking on
  Windows, for which I did not test); also fixed the $UID issue
  mentioned in review

* extended xmllint testing to the XML files generated for `schtasks`
  on Windows too; this required a small modification to the XML header
  boilerplate to specify the correct file encoding since `xmllint`
  complains when the file is UTF-8 but claims to be UTF-16; now that
  the test script captures the generated `schtasks` XML file for
  checking against `xmllint`, you have the opportunity to perform
  other sorts of validation checks on the XML too, such as you do in
  the MacOS `launchctl` test (though I did not add any additional
  checks)

* fixed a potentially crashable `fprintf(xmlfp, xml)` by changing it
  to `fputs(xml, xmlfp)` since the compiler complains about the former
  because it can crash if `xml` contains a "%"

* fixed the malformed write_script() issue for the MacOS test
  mentioned in review

--- >8 ---
From 016887b9fa4269bd4df46bea1d7849c08aba6ad6 Mon Sep 17 00:00:00 2001
From: Eric Sunshine <sunshine@sunshineco.com>
Date: Sat, 14 Nov 2020 02:39:05 -0500
Subject: [PATCH] maintenance: test start/stop on all platforms from any
 platform

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 builtin/gc.c           | 204 +++++++++++++++++++----------------------
 t/t7900-maintenance.sh |  66 +++++++++----
 t/test-lib.sh          |   4 -
 3 files changed, 143 insertions(+), 131 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index 76a3afa20a..955d4b3baf 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,16 +1491,28 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
-#if defined(__APPLE__)
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
 
-static char *get_service_name(const char *frequency)
+static char *launchctl_service_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
 	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
 	return strbuf_detach(&label, NULL);
 }
 
-static char *get_service_filename(const char *name)
+static char *launchctl_service_filename(const char *name)
 {
 	char *expanded;
 	struct strbuf filename = STRBUF_INIT;
@@ -1514,49 +1526,23 @@ static char *get_service_filename(const char *name)
 	return expanded;
 }
 
-static const char *get_frequency(enum schedule_priority schedule)
-{
-	switch (schedule) {
-	case SCHEDULE_HOURLY:
-		return "hourly";
-	case SCHEDULE_DAILY:
-		return "daily";
-	case SCHEDULE_WEEKLY:
-		return "weekly";
-	default:
-		BUG("invalid schedule %d", schedule);
-	}
-}
-
-static char *get_uid(void)
+static char *launchctl_get_uid(void)
 {
-	struct strbuf output = STRBUF_INIT;
-	struct child_process id = CHILD_PROCESS_INIT;
-
-	strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL);
-	if (capture_command(&id, &output, 0))
-		die(_("failed to discover user id"));
-
-	strbuf_trim_trailing_newline(&output);
-	return strbuf_detach(&output, NULL);
+	return xstrfmt("gui/%d", getuid());
 }
 
-static int boot_plist(int enable, const char *filename)
+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
 {
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
-	char *uid = get_uid();
-	const char *launchctl = getenv("GIT_TEST_CRONTAB");
-	if (!launchctl)
-		launchctl = "/bin/launchctl";
-
-	strvec_split(&child.args, launchctl);
+	char *uid = launchctl_get_uid();
 
+	strvec_split(&child.args, cmd);
 	if (enable)
 		strvec_push(&child.args, "bootstrap");
 	else
 		strvec_push(&child.args, "bootout");
-	strvec_pushf(&child.args, "gui/%s", uid);
+	strvec_push(&child.args, uid);
 	strvec_push(&child.args, filename);
 
 	child.no_stderr = 1;
@@ -1571,33 +1557,33 @@ static int boot_plist(int enable, const char *filename)
 	return result;
 }
 
-static int remove_plist(enum schedule_priority schedule)
+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
 {
 	const char *frequency = get_frequency(schedule);
-	char *name = get_service_name(frequency);
-	char *filename = get_service_filename(name);
-	int result = boot_plist(0, filename);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+	int result = launchctl_boot_plist(0, filename, cmd);
 	unlink(filename);
 	free(filename);
 	free(name);
 	return result;
 }
 
-static int remove_plists(void)
+static int launchctl_remove_plists(const char *cmd)
 {
-	return remove_plist(SCHEDULE_HOURLY) ||
-		remove_plist(SCHEDULE_DAILY) ||
-		remove_plist(SCHEDULE_WEEKLY);
+	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
 }
 
-static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
 {
 	FILE *plist;
 	int i;
 	const char *preamble, *repeat;
 	const char *frequency = get_frequency(schedule);
-	char *name = get_service_name(frequency);
-	char *filename = get_service_filename(name);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
 
 	if (safe_create_leading_directories(filename))
 		die(_("failed to create directories for '%s'"), filename);
@@ -1658,8 +1644,8 @@ static int schedule_plist(const char *exec_path, enum schedule_priority schedule
 	fprintf(plist, "</array>\n</dict>\n</plist>\n");
 
 	/* bootout might fail if not already running, so ignore */
-	boot_plist(0, filename);
-	if (boot_plist(1, filename))
+	launchctl_boot_plist(0, filename, cmd);
+	if (launchctl_boot_plist(1, filename, cmd))
 		die(_("failed to bootstrap service %s"), filename);
 
 	fclose(plist);
@@ -1668,57 +1654,38 @@ static int schedule_plist(const char *exec_path, enum schedule_priority schedule
 	return 0;
 }
 
-static int add_plists(void)
+static int launchctl_add_plists(const char *cmd)
 {
 	const char *exec_path = git_exec_path();
 
-	return schedule_plist(exec_path, SCHEDULE_HOURLY) ||
-		schedule_plist(exec_path, SCHEDULE_DAILY) ||
-		schedule_plist(exec_path, SCHEDULE_WEEKLY);
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int platform_update_schedule(int run_maintenance, int fd)
+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	if (run_maintenance)
-		return add_plists();
+		return launchctl_add_plists(cmd);
 	else
-		return remove_plists();
-}
-
-#elif defined(GIT_WINDOWS_NATIVE)
-
-static const char *get_frequency(enum schedule_priority schedule)
-{
-	switch (schedule) {
-	case SCHEDULE_HOURLY:
-		return "hourly";
-	case SCHEDULE_DAILY:
-		return "daily";
-	case SCHEDULE_WEEKLY:
-		return "weekly";
-	default:
-		BUG("invalid schedule %d", schedule);
-	}
+		return launchctl_remove_plists(cmd);
 }
 
-static char *get_task_name(const char *frequency)
+static char *schtasks_task_name(const char *frequency)
 {
 	struct strbuf label = STRBUF_INIT;
 	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
 	return strbuf_detach(&label, NULL);
 }
 
-static int remove_task(enum schedule_priority schedule)
+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
 {
 	int result;
 	struct strvec args = STRVEC_INIT;
 	const char *frequency = get_frequency(schedule);
-	char *name = get_task_name(frequency);
-	const char *schtasks = getenv("GIT_TEST_CRONTAB");
-	if (!schtasks)
-		schtasks = "schtasks";
+	char *name = schtasks_task_name(frequency);
 
-	strvec_split(&args, schtasks);
+	strvec_split(&args, cmd);
 	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
 
 	result = run_command_v_opt(args.v, 0);
@@ -1728,33 +1695,33 @@ static int remove_task(enum schedule_priority schedule)
 	return result;
 }
 
-static int remove_scheduled_tasks(void)
+static int schtasks_remove_tasks(const char *cmd)
 {
-	return remove_task(SCHEDULE_HOURLY) ||
-		remove_task(SCHEDULE_DAILY) ||
-		remove_task(SCHEDULE_WEEKLY);
+	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
+		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
 }
 
-static int schedule_task(const char *exec_path, enum schedule_priority schedule)
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
 {
 	int result;
 	struct child_process child = CHILD_PROCESS_INIT;
-	const char *xml, *schtasks;
+	const char *xml;
 	char *xmlpath, *tempDir;
 	FILE *xmlfp;
 	const char *frequency = get_frequency(schedule);
-	char *name = get_task_name(frequency);
+	char *name = schtasks_task_name(frequency);
 
 	tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
 	xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
 	safe_create_leading_directories(xmlpath);
 	xmlfp = xfopen(xmlpath, "w");
 
-	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
+	xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
 	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
 	      "<Triggers>\n"
 	      "<CalendarTrigger>\n";
-	fprintf(xmlfp, xml);
+	fputs(xml, xmlfp);
 
 	switch (schedule) {
 	case SCHEDULE_HOURLY:
@@ -1831,10 +1798,7 @@ static int schedule_task(const char *exec_path, enum schedule_priority schedule)
 	fprintf(xmlfp, xml, exec_path, exec_path, frequency);
 	fclose(xmlfp);
 
-	schtasks = getenv("GIT_TEST_CRONTAB");
-	if (!schtasks)
-		schtasks = "schtasks";
-	strvec_split(&child.args, schtasks);
+	strvec_split(&child.args, cmd);
 	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
 
 	child.no_stdout = 1;
@@ -1851,42 +1815,36 @@ static int schedule_task(const char *exec_path, enum schedule_priority schedule)
 	return result;
 }
 
-static int add_scheduled_tasks(void)
+static int schtasks_schedule_tasks(const char *cmd)
 {
 	const char *exec_path = git_exec_path();
 
-	return schedule_task(exec_path, SCHEDULE_HOURLY) ||
-		schedule_task(exec_path, SCHEDULE_DAILY) ||
-		schedule_task(exec_path, SCHEDULE_WEEKLY);
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
 }
 
-static int platform_update_schedule(int run_maintenance, int fd)
+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	if (run_maintenance)
-		return add_scheduled_tasks();
+		return schtasks_schedule_tasks(cmd);
 	else
-		return remove_scheduled_tasks();
+		return schtasks_remove_tasks(cmd);
 }
 
-#else
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int platform_update_schedule(int run_maintenance, int fd)
+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
 	FILE *cron_list, *cron_in;
-	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
 
-	crontab_name = getenv("GIT_TEST_CRONTAB");
-	if (!crontab_name)
-		crontab_name = "crontab";
-
-	strvec_split(&crontab_list.args, crontab_name);
+	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
 	crontab_list.out = dup(fd);
@@ -1906,7 +1864,7 @@ static int platform_update_schedule(int run_maintenance, int fd)
 	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
-	strvec_split(&crontab_edit.args, crontab_name);
+	strvec_split(&crontab_edit.args, cmd);
 	crontab_edit.in = -1;
 	crontab_edit.git_cmd = 0;
 
@@ -1963,20 +1921,48 @@ static int platform_update_schedule(int run_maintenance, int fd)
 		fclose(cron_list);
 	return result;
 }
+
+#if defined(__APPLE__)
+static const char platform_scheduler[] = "launchctl";
+#elif defined(GIT_WINDOWS_NATIVE)
+static const char platform_scheduler[] = "schtasks";
+#else
+static const char platform_scheduler[] = "crontab";
 #endif
 
-static int update_background_schedule(int run_maintenance)
+static int update_background_schedule(int enable)
 {
 	int result;
+	const char *scheduler = platform_scheduler;
+	const char *cmd = scheduler;
+	char *testing;
 	struct lock_file lk;
 	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
+	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	if (testing) {
+		char *sep = strchr(testing, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+		scheduler = testing;
+		cmd = sep + 1;
+	}
+
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	result = platform_update_schedule(run_maintenance, lk.tempfile->fd);
+	if (!strcmp(scheduler, "launchctl"))
+		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "schtasks"))
+		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "crontab"))
+		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
+	else
+		die("unknown background scheduler: %s", scheduler);
 
 	rollback_lock_file(&lk);
+	free(testing);
 	return result;
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 0dc2479117..e92946c10a 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -7,6 +7,19 @@ test_description='git maintenance builtin'
 GIT_TEST_COMMIT_GRAPH=0
 GIT_TEST_MULTI_PACK_INDEX=0
 
+test_lazy_prereq XMLLINT '
+	xmllint --version
+'
+
+test_xmllint () {
+	if test_have_prereq XMLLINT
+	then
+		xmllint --noout "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -367,8 +380,8 @@ test_expect_success 'register and unregister' '
 	test_cmp before actual
 '
 
-test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+test_expect_success 'start from empty cron table' '
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -378,28 +391,32 @@ test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+test_expect_success 'stop from existing schedule' '
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
 	# Operation is idempotent
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 	test_must_be_empty cron.txt
 '
 
-test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '
+test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
-test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
-	write_script print-args "#!/bin/sh\necho \$* >>args" &&
+test_expect_success 'start and stop macOS maintenance' '
+	uid=$(id -u) &&
+
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
 
 	rm -f args &&
-	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -417,33 +434,41 @@ test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
 	for frequency in hourly daily weekly
 	do
 		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
-		xmllint --noout "$PLIST" &&
+		test_xmllint "$PLIST" &&
 		grep schedule=$frequency "$PLIST" &&
-		echo "bootout gui/$UID $PLIST" >>expect &&
-		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
+		echo "bootout gui/$uid $PLIST" >>expect &&
+		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
 	done &&
 	test_cmp expect args &&
 
 	rm -f args &&
-	GIT_TEST_CRONTAB="./print-args" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
-	printf "bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
 		hourly daily weekly >expect &&
 	test_cmp expect args &&
 	ls "$HOME/Library/LaunchAgents" >actual &&
 	test_line_count = 0 actual
 '
 
-test_expect_success MINGW 'start and stop Windows maintenance' '
+test_expect_success 'start and stop Windows maintenance' '
 	write_script print-args <<-\EOF &&
 	echo $* >>args
+	while test $# -gt 0
+	do
+		case "$1" in
+		/xml) shift; xmlfile=$1; break ;;
+		*) shift ;;
+		esac
+	done
+	test -z "$xmlfile" || cp "$xmlfile" .
 	EOF
 
 	rm -f args &&
-	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:/bin/sh print-args" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -452,8 +477,13 @@ test_expect_success MINGW 'start and stop Windows maintenance' '
 		hourly hourly daily daily weekly weekly >expect &&
 	test_cmp expect args &&
 
+	for frequency in hourly daily weekly
+	do
+		test_xmllint "schedule-$frequency.xml"
+	done &&
+
 	rm -f args &&
-	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:/bin/sh print-args" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 620ffbf3af..4a60d1ed76 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1703,10 +1703,6 @@ test_lazy_prereq REBASE_P '
 	test -z "$GIT_TEST_SKIP_REBASE_P"
 '
 
-test_lazy_prereq MACOS_MAINTENANCE '
-	launchctl list
-'
-
 # Ensure that no test accidentally triggers a Git command
 # that runs 'crontab', affecting a user's cron schedule.
 # Tests that verify the cron integration must set this locally
-- 
2.29.2.454.gaff20da3a2

Re: [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 11/14/2020 4:23 AM, Eric Sunshine wrote:
> On Fri, Nov 13, 2020 at 03:47:15PM -0500, Eric Sunshine wrote:
>> I forgot to mention a couple things when reviewing the patches
>> individually, so I'll point them out here...
> 
> In v2, you added an `xmllint` check on MacOS after discovering that
> gc.c was generating a malformed .plist file on that platform. That got
> me thinking that it would have been nice to have caught the problem
> earlier, if possible, even without having access to MacOS. Since none
> of the code added to gc.c has a hard platform dependency, it should be
> possible to perform all the tests on any platform rather than
> restricting them to specific platforms via test prerequisites. The
> patch below, which is built atop v3, does just that. It removes the
> conditional compilation directives from gc.c and the prerequisites
> from the test script so that all scheduler-specific code in gc.c is
> tested on all platform.
> 
> The changes made by the patch are intended to be folded into each of
> your patches where appropriate (rather than existing atop your series,
> which, though possible, would be ugly). If you're interested in
> incorporating any of these improvements into v4, you can have my
> "Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>" in addition
> to the Helped-by: you already added.

This approach is fascinating. I will tease it apart to appropriately
incorporate it into my series. Thank you for your sign-off, since
this elevates the patches from "Helped-by" to "Co-authored by".

Thanks,
-Stolee

[PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.

Updates in V4
=============

 * Eric did an excellent job providing a patch that cleans up several parts
   of my series. The most impressive is his mechanism for testing the
   platform-specific Git logic in a way that is (mostly) platform-agnostic.
   
   
 * Windows doesn't have the 'id' command, so we cannot run the macOS
   platform test on Windows.
   
   
 * I noticed far too late that while my example XML files had been edited
   with UTF-8 encoding, Git is actually writing them as US-ASCII. Somehow 
   xmllint and launchd are not complaining, but schtasks does complain.
   Unfortunately, I cannot find a way to catch this problem other than to
   install my tip version on all three platforms and go through the entire 
   git maintenance start process, and double-check that the processes are
   running on the hour.
   
   

Here is a diff from the tip of v3 + Eric's patch to the tip of v4:

diff --git a/builtin/gc.c b/builtin/gc.c
index 955d4b3baf..1a3725429c 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1642,13 +1642,13 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
         break;
     }
     fprintf(plist, "</array>\n</dict>\n</plist>\n");
+    fclose(plist);

     /* bootout might fail if not already running, so ignore */
     launchctl_boot_plist(0, filename, cmd);
     if (launchctl_boot_plist(1, filename, cmd))
         die(_("failed to bootstrap service %s"), filename);

-    fclose(plist);
     free(filename);
     free(name);
     return 0;
@@ -1707,25 +1707,27 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
     int result;
     struct child_process child = CHILD_PROCESS_INIT;
     const char *xml;
-    char *xmlpath, *tempDir;
-    FILE *xmlfp;
+    char *xmlpath;
+    struct tempfile *tfile;
     const char *frequency = get_frequency(schedule);
     char *name = schtasks_task_name(frequency);

-    tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
-    xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
-    safe_create_leading_directories(xmlpath);
-    xmlfp = xfopen(xmlpath, "w");
+    xmlpath =  xstrfmt("%s/schedule-%s.xml",
+               the_repository->objects->odb->path,
+               frequency);
+    tfile = create_tempfile(xmlpath);
+    if (!tfile || !fdopen_tempfile(tfile, "w"))
+        die(_("failed to create '%s'"), xmlpath);

-    xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+    xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
           "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
           "<Triggers>\n"
           "<CalendarTrigger>\n";
-    fputs(xml, xmlfp);
+    fputs(xml, tfile->fp);

     switch (schedule) {
     case SCHEDULE_HOURLY:
-        fprintf(xmlfp,
+        fprintf(tfile->fp,
             "<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
             "<Enabled>true</Enabled>\n"
             "<ScheduleByDay>\n"
@@ -1739,7 +1741,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
         break;

     case SCHEDULE_DAILY:
-        fprintf(xmlfp,
+        fprintf(tfile->fp,
             "<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
             "<Enabled>true</Enabled>\n"
             "<ScheduleByWeek>\n"
@@ -1756,7 +1758,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
         break;

     case SCHEDULE_WEEKLY:
-        fprintf(xmlfp,
+        fprintf(tfile->fp,
             "<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
             "<Enabled>true</Enabled>\n"
             "<ScheduleByWeek>\n"
@@ -1771,7 +1773,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
         break;
     }

-    xml=  "</CalendarTrigger>\n"
+    xml = "</CalendarTrigger>\n"
           "</Triggers>\n"
           "<Principals>\n"
           "<Principal id=\"Author\">\n"
@@ -1795,11 +1797,10 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
           "</Exec>\n"
           "</Actions>\n"
           "</Task>\n";
-    fprintf(xmlfp, xml, exec_path, exec_path, frequency);
-    fclose(xmlfp);
-
+    fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
     strvec_split(&child.args, cmd);
     strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
+    close_tempfile_gently(tfile);

     child.no_stdout = 1;
     child.no_stderr = 1;
@@ -1808,8 +1809,7 @@ static int schtasks_schedule_task(const char *exec_path, enum schedule_priority
         die(_("failed to start schtasks"));
     result = finish_command(&child);

-    unlink(xmlpath);
-    rmdir(tempDir);
+    delete_tempfile(&tfile);
     free(xmlpath);
     free(name);
     return result;
@@ -1850,9 +1850,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
     crontab_list.out = dup(fd);
     crontab_list.git_cmd = 0;

-    if (start_command(&crontab_list)) {
+    if (start_command(&crontab_list))
         return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-    }

     /* Ignore exit code, as an empty crontab will return error. */
     finish_command(&crontab_list);
@@ -1868,9 +1867,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
     crontab_edit.in = -1;
     crontab_edit.git_cmd = 0;

-    if (start_command(&crontab_edit)) {
+    if (start_command(&crontab_edit))
         return error(_("failed to run 'crontab'; your system might not support 'cron'"));
-    }

     cron_in = fdopen(crontab_edit.in, "w");
     if (!cron_in) {
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index e92946c10a..a26ff22541 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -408,7 +408,7 @@ test_expect_success 'start preserves existing schedule' '
     grep "Important information!" cron.txt
 '

-test_expect_success 'start and stop macOS maintenance' '
+test_expect_success !MINGW 'start and stop macOS maintenance' '
     uid=$(id -u) &&

     write_script print-args <<-\EOF &&
@@ -421,7 +421,6 @@ test_expect_success 'start and stop macOS maintenance' '
     # start registers the repo
     git config --get --global maintenance.repo "$(pwd)" &&

-    # ~/Library/LaunchAgents
     ls "$HOME/Library/LaunchAgents" >actual &&
     cat >expect <<-\EOF &&
     org.git-scm.git.daily.plist
@@ -468,12 +467,12 @@ test_expect_success 'start and stop Windows maintenance' '
     EOF

     rm -f args &&
-    GIT_TEST_MAINT_SCHEDULER="schtasks:/bin/sh print-args" git maintenance start &&
+    GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&

     # start registers the repo
     git config --get --global maintenance.repo "$(pwd)" &&

-    printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\n" \
+    printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
         hourly hourly daily daily weekly weekly >expect &&
     test_cmp expect args &&

@@ -483,7 +482,7 @@ test_expect_success 'start and stop Windows maintenance' '
     done &&

     rm -f args &&
-    GIT_TEST_MAINT_SCHEDULER="schtasks:/bin/sh print-args" git maintenance stop &&
+    GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&

     # stop does not unregister the repo
     git config --get --global maintenance.repo "$(pwd)" &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..ddbeee1f5e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '
 '

 # Ensure that no test accidentally triggers a Git command
-# that runs 'crontab', affecting a user's cron schedule.
-# Tests that verify the cron integration must set this locally
+# that runs the actual maintenance scheduler, affecting a user's
+# system permanently.
+# Tests that verify the scheduler integration must set this locally
 # to avoid errors.
-GIT_TEST_CRONTAB="exit 1"
+GIT_TEST_MAINT_SCHEDULER="none:exit 1"

Thanks, -Stolee

cc: jrnieder@gmail.com [jrnieder@gmail.com], jonathantanmy@google.com
[jonathantanmy@google.com], sluongng@gmail.com [sluongng@gmail.com]cc:
Derrick Stolee stolee@gmail.com [stolee@gmail.com]cc: Đoàn Trần Công Danh 
congdanhqx@gmail.com [congdanhqx@gmail.com]cc: Martin Ågren 
martin.agren@gmail.com [martin.agren@gmail.com]cc: Eric Sunshine 
sunshine@sunshineco.com [sunshine@sunshineco.com]cc: Derrick Stolee 
stolee@gmail.com [stolee@gmail.com]

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 116 ++++++++
 builtin/gc.c                      | 421 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 106 +++++++-
 t/test-lib.sh                     |   7 +-
 4 files changed, 616 insertions(+), 34 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v3:

 1:  d35f1aa162 ! 1:  4807342b00 maintenance: extract platform-specific scheduling
     @@ Commit message
          swapped at compile time with new implementations on specialized
          platforms.
      
     +    As we add this generality, rename GIT_TEST_CRONTAB to
     +    GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as
     +    "<scheduler>:<command>" so we can test platform-specific scheduling
     +    logic even when not on the correct platform. By specifying the
     +    <scheduler> in this string, we will be able to test all three sets of
     +    Git logic from a Linux machine.
     +
     +    Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
     +    Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## builtin/gc.c ##
     @@ builtin/gc.c: static int maintenance_unregister(void)
       #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
       
      -static int update_background_schedule(int run_maintenance)
     -+static int platform_update_schedule(int run_maintenance, int fd)
     ++static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
       {
       	int result = 0;
       	int in_old_region = 0;
     -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
     + 	struct child_process crontab_list = CHILD_PROCESS_INIT;
     + 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
       	FILE *cron_list, *cron_in;
     - 	const char *crontab_name;
     +-	const char *crontab_name;
       	struct strbuf line = STRBUF_INIT;
      -	struct lock_file lk;
      -	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
     --
     + 
      -	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
      -		return error(_("another process is scheduling background maintenance"));
     - 
     - 	crontab_name = getenv("GIT_TEST_CRONTAB");
     - 	if (!crontab_name)
     -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
     - 	strvec_split(&crontab_list.args, crontab_name);
     +-
     +-	crontab_name = getenv("GIT_TEST_CRONTAB");
     +-	if (!crontab_name)
     +-		crontab_name = "crontab";
     +-
     +-	strvec_split(&crontab_list.args, crontab_name);
     ++	strvec_split(&crontab_list.args, cmd);
       	strvec_push(&crontab_list.args, "-l");
       	crontab_list.in = -1;
      -	crontab_list.out = dup(lk.tempfile->fd);
      +	crontab_list.out = dup(fd);
       	crontab_list.git_cmd = 0;
       
     - 	if (start_command(&crontab_list)) {
     +-	if (start_command(&crontab_list)) {
      -		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
      -		goto cleanup;
     +-	}
     ++	if (start_command(&crontab_list))
      +		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
     - 	}
       
       	/* Ignore exit code, as an empty crontab will return error. */
     + 	finish_command(&crontab_list);
      @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
       	 * Read from the .lock file, filtering out the old
       	 * schedule while appending the new schedule.
     @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
      +	cron_list = fdopen(fd, "r");
       	rewind(cron_list);
       
     - 	strvec_split(&crontab_edit.args, crontab_name);
     -@@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
     +-	strvec_split(&crontab_edit.args, crontab_name);
     ++	strvec_split(&crontab_edit.args, cmd);
     + 	crontab_edit.in = -1;
       	crontab_edit.git_cmd = 0;
       
     - 	if (start_command(&crontab_edit)) {
     +-	if (start_command(&crontab_edit)) {
      -		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
      -		goto cleanup;
     +-	}
     ++	if (start_command(&crontab_edit))
      +		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
     - 	}
       
       	cron_in = fdopen(crontab_edit.in, "w");
     + 	if (!cron_in) {
      @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
       	close(crontab_edit.in);
       
     @@ builtin/gc.c: static int update_background_schedule(int run_maintenance)
      +	if (finish_command(&crontab_edit))
       		result = error(_("'crontab' died"));
      -		goto cleanup;
     --	}
     --	fclose(cron_list);
      +	else
      +		fclose(cron_list);
      +	return result;
      +}
      +
     -+static int update_background_schedule(int run_maintenance)
     ++static const char platform_scheduler[] = "crontab";
     ++
     ++static int update_background_schedule(int enable)
      +{
      +	int result;
     ++	const char *scheduler = platform_scheduler;
     ++	const char *cmd = scheduler;
     ++	char *testing;
      +	struct lock_file lk;
      +	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
      +
     ++	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
     ++	if (testing) {
     ++		char *sep = strchr(testing, ':');
     ++		if (!sep)
     ++			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
     ++		*sep = '\0';
     ++		scheduler = testing;
     ++		cmd = sep + 1;
     + 	}
     +-	fclose(cron_list);
     + 
     +-cleanup:
      +	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
      +		return error(_("another process is scheduling background maintenance"));
      +
     -+	result = platform_update_schedule(run_maintenance, lk.tempfile->fd);
     - 
     --cleanup:
     ++	if (!strcmp(scheduler, "crontab"))
     ++		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
     ++	else
     ++		die("unknown background scheduler: %s", scheduler);
     ++
       	rollback_lock_file(&lk);
     ++	free(testing);
       	return result;
       }
     + 
     +
     + ## t/t7900-maintenance.sh ##
     +@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '
     + '
     + 
     + test_expect_success 'start from empty cron table' '
     +-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
     + 
     + 	# start registers the repo
     + 	git config --get --global maintenance.repo "$(pwd)" &&
     +@@ t/t7900-maintenance.sh: test_expect_success 'start from empty cron table' '
     + '
     + 
     + test_expect_success 'stop from existing schedule' '
     +-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
     ++	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
     + 
     + 	# stop does not unregister the repo
     + 	git config --get --global maintenance.repo "$(pwd)" &&
     + 
     + 	# Operation is idempotent
     +-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
     ++	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
     + 	test_must_be_empty cron.txt
     + '
     + 
     + test_expect_success 'start preserves existing schedule' '
     + 	echo "Important information!" >cron.txt &&
     +-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
     + 	grep "Important information!" cron.txt
     + '
     + 
     +
     + ## t/test-lib.sh ##
     +@@ t/test-lib.sh: test_lazy_prereq REBASE_P '
     + '
     + 
     + # Ensure that no test accidentally triggers a Git command
     +-# that runs 'crontab', affecting a user's cron schedule.
     +-# Tests that verify the cron integration must set this locally
     ++# that runs the actual maintenance scheduler, affecting a user's
     ++# system permanently.
     ++# Tests that verify the scheduler integration must set this locally
     + # to avoid errors.
     +-GIT_TEST_CRONTAB="exit 1"
     ++GIT_TEST_MAINT_SCHEDULER="none:exit 1"
 2:  0dfe53092e ! 2:  99170df462 maintenance: include 'cron' details in docs
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +---------------------------------------
      +
      +The standard mechanism for scheduling background tasks on POSIX systems
     -+is cron(8). This tool executes commands based on a given schedule. The
     ++is `cron`. This tool executes commands based on a given schedule. The
      +current list of user-scheduled tasks can be found by running `crontab -l`.
      +The schedule written by `git maintenance start` is similar to this:
      +
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +Any modifications within this region will be completely deleted by
      +`git maintenance stop` or overwritten by `git maintenance start`.
      +
     -+The `crontab` entry specifies the full path of the `git` executable to
     -+ensure that the executed `git` command is the same one with which
     -+`git maintenance start` was issued independent of `PATH`. If the same user
     -+runs `git maintenance start` with multiple Git executables, then only the
     -+latest executable is used.
     ++The `<path>` string is loaded to specifically use the location for the
     ++`git` executable used in the `git maintenance start` command. This allows
     ++for multiple versions to be compatible. However, if the same user runs
     ++`git maintenance start` with multiple Git executables, then only the
     ++latest executable will be used.
      +
      +These commands use `git for-each-repo --config=maintenance.repo` to run
      +`git maintenance run --schedule=<frequency>` on each repository listed in
      +the multi-valued `maintenance.repo` config option. These are typically
     -+loaded from the user-specific global config. The `git maintenance` process
     -+then determines which maintenance tasks are configured to run on each
     -+repository with each `<frequency>` using the `maintenance.<task>.schedule`
     -+config options. These values are loaded from the global or repository
     -+config values.
     ++loaded from the user-specific global config located at `~/.gitconfig`.
     ++The `git maintenance` process then determines which maintenance tasks
     ++are configured to run on each repository with each `<frequency>` using
     ++the `maintenance.<task>.schedule` config options. These values are loaded
     ++from the global or repository config values.
      +
      +If the config values are insufficient to achieve your desired background
      +maintenance schedule, then you can create your own schedule. If you run
      +`crontab -e`, then an editor will load with your user-specific `cron`
      +schedule. In that editor, you can add your own schedule lines. You could
      +start by adapting the default schedule listed earlier, or you could read
     -+the crontab(5) documentation for advanced scheduling techniques. Please
     -+do use the full path and `--exec-path` techniques from the default
     -+schedule to ensure you are executing the correct binaries in your
     -+schedule.
     ++https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
     ++for advanced scheduling techniques. Please do use the full path and
     ++`--exec-path` techniques from the default schedule to ensure you are
     ++executing the correct binaries in your schedule.
      +
       
       GIT
 3:  1629bcfcf8 ! 3:  ed0a0011fb maintenance: use launchctl on macOS
     @@ Commit message
          plist file. We also need to 'bootout' a task before the 'bootstrap'
          subcommand will succeed, if such a task already exists.
      
     +    The need for a user id requires us to run 'id -u' which works on
     +    POSIX systems but not Windows. The test therefore has a prerequisite
     +    that we are not on Windows. The cross-platform logic still allows us to
     +    test the macOS logic on a Linux machine.
     +
          We can verify the commands that were run by 'git maintenance start'
          and 'git maintenance stop' by injecting a script that writes the
     -    command-line arguments into GIT_TEST_CRONTAB.
     +    command-line arguments into GIT_TEST_MAINT_SCHEDULER.
      
          An earlier version of this patch accidentally had an opening
          "<dict>" tag when it should have had a closing "</dict>" tag. This
          was caught during manual testing with actual 'launchctl' commands,
          but we do not want to update developers' tasks when running tests.
          It appears that macOS includes the "xmllint" tool which can verify
     -    the XML format, so call it from the macOS-specific tests to ensure
     -    the .plist files are well-formatted.
     +    the XML format. This is useful for any system that might contain
     +    the tool, so use it whenever it is available.
      
     -    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
     +    Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
     +    Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     -@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your
     - schedule.
     +@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
     + executing the correct binaries in your schedule.
       
       
      +BACKGROUND MAINTENANCE ON MACOS SYSTEMS
     @@ builtin/gc.c: static int maintenance_unregister(void)
       	return run_command(&config_unset);
       }
       
     -+#if defined(__APPLE__)
     ++static const char *get_frequency(enum schedule_priority schedule)
     ++{
     ++	switch (schedule) {
     ++	case SCHEDULE_HOURLY:
     ++		return "hourly";
     ++	case SCHEDULE_DAILY:
     ++		return "daily";
     ++	case SCHEDULE_WEEKLY:
     ++		return "weekly";
     ++	default:
     ++		BUG("invalid schedule %d", schedule);
     ++	}
     ++}
      +
     -+static char *get_service_name(const char *frequency)
     ++static char *launchctl_service_name(const char *frequency)
      +{
      +	struct strbuf label = STRBUF_INIT;
      +	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
      +	return strbuf_detach(&label, NULL);
      +}
      +
     -+static char *get_service_filename(const char *name)
     ++static char *launchctl_service_filename(const char *name)
      +{
      +	char *expanded;
      +	struct strbuf filename = STRBUF_INIT;
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +	return expanded;
      +}
      +
     -+static const char *get_frequency(enum schedule_priority schedule)
     ++static char *launchctl_get_uid(void)
      +{
     -+	switch (schedule) {
     -+	case SCHEDULE_HOURLY:
     -+		return "hourly";
     -+	case SCHEDULE_DAILY:
     -+		return "daily";
     -+	case SCHEDULE_WEEKLY:
     -+		return "weekly";
     -+	default:
     -+		BUG("invalid schedule %d", schedule);
     -+	}
     ++	return xstrfmt("gui/%d", getuid());
      +}
      +
     -+static char *get_uid(void)
     -+{
     -+	struct strbuf output = STRBUF_INIT;
     -+	struct child_process id = CHILD_PROCESS_INIT;
     -+
     -+	strvec_pushl(&id.args, "/usr/bin/id", "-u", NULL);
     -+	if (capture_command(&id, &output, 0))
     -+		die(_("failed to discover user id"));
     -+
     -+	strbuf_trim_trailing_newline(&output);
     -+	return strbuf_detach(&output, NULL);
     -+}
     -+
     -+static int boot_plist(int enable, const char *filename)
     ++static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
      +{
      +	int result;
      +	struct child_process child = CHILD_PROCESS_INIT;
     -+	char *uid = get_uid();
     -+	const char *launchctl = getenv("GIT_TEST_CRONTAB");
     -+	if (!launchctl)
     -+		launchctl = "/bin/launchctl";
     -+
     -+	strvec_split(&child.args, launchctl);
     ++	char *uid = launchctl_get_uid();
      +
     ++	strvec_split(&child.args, cmd);
      +	if (enable)
      +		strvec_push(&child.args, "bootstrap");
      +	else
      +		strvec_push(&child.args, "bootout");
     -+	strvec_pushf(&child.args, "gui/%s", uid);
     ++	strvec_push(&child.args, uid);
      +	strvec_push(&child.args, filename);
      +
      +	child.no_stderr = 1;
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +	return result;
      +}
      +
     -+static int remove_plist(enum schedule_priority schedule)
     ++static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
      +{
      +	const char *frequency = get_frequency(schedule);
     -+	char *name = get_service_name(frequency);
     -+	char *filename = get_service_filename(name);
     -+	int result = boot_plist(0, filename);
     ++	char *name = launchctl_service_name(frequency);
     ++	char *filename = launchctl_service_filename(name);
     ++	int result = launchctl_boot_plist(0, filename, cmd);
      +	unlink(filename);
      +	free(filename);
      +	free(name);
      +	return result;
      +}
      +
     -+static int remove_plists(void)
     ++static int launchctl_remove_plists(const char *cmd)
      +{
     -+	return remove_plist(SCHEDULE_HOURLY) ||
     -+		remove_plist(SCHEDULE_DAILY) ||
     -+		remove_plist(SCHEDULE_WEEKLY);
     ++	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
     ++		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
     ++		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
      +}
      +
     -+static int schedule_plist(const char *exec_path, enum schedule_priority schedule)
     ++static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
      +{
      +	FILE *plist;
      +	int i;
      +	const char *preamble, *repeat;
      +	const char *frequency = get_frequency(schedule);
     -+	char *name = get_service_name(frequency);
     -+	char *filename = get_service_filename(name);
     ++	char *name = launchctl_service_name(frequency);
     ++	char *filename = launchctl_service_filename(name);
      +
      +	if (safe_create_leading_directories(filename))
      +		die(_("failed to create directories for '%s'"), filename);
      +	plist = xfopen(filename, "w");
      +
     -+	preamble = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
     ++	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
      +		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
      +		   "<plist version=\"1.0\">"
      +		   "<dict>\n"
     @@ builtin/gc.c: static int maintenance_unregister(void)
      +		break;
      +	}
      +	fprintf(plist, "</array>\n</dict>\n</plist>\n");
     ++	fclose(plist);
      +
      +	/* bootout might fail if not already running, so ignore */
     -+	boot_plist(0, filename);
     -+	if (boot_plist(1, filename))
     ++	launchctl_boot_plist(0, filename, cmd);
     ++	if (launchctl_boot_plist(1, filename, cmd))
      +		die(_("failed to bootstrap service %s"), filename);
      +
     -+	fclose(plist);
      +	free(filename);
      +	free(name);
      +	return 0;
      +}
      +
     -+static int add_plists(void)
     ++static int launchctl_add_plists(const char *cmd)
      +{
      +	const char *exec_path = git_exec_path();
      +
     -+	return schedule_plist(exec_path, SCHEDULE_HOURLY) ||
     -+		schedule_plist(exec_path, SCHEDULE_DAILY) ||
     -+		schedule_plist(exec_path, SCHEDULE_WEEKLY);
     ++	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
     ++		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
     ++		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
      +}
      +
     -+static int platform_update_schedule(int run_maintenance, int fd)
     ++static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
      +{
      +	if (run_maintenance)
     -+		return add_plists();
     ++		return launchctl_add_plists(cmd);
      +	else
     -+		return remove_plists();
     ++		return launchctl_remove_plists(cmd);
      +}
     -+#else
     ++
       #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
       #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
       
     -@@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
     - 		fclose(cron_list);
     +@@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
       	return result;
       }
     + 
     ++#if defined(__APPLE__)
     ++static const char platform_scheduler[] = "launchctl";
     ++#else
     + static const char platform_scheduler[] = "crontab";
      +#endif
       
     - static int update_background_schedule(int run_maintenance)
     + static int update_background_schedule(int enable)
       {
     +@@ builtin/gc.c: static int update_background_schedule(int enable)
     + 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
     + 		return error(_("another process is scheduling background maintenance"));
     + 
     +-	if (!strcmp(scheduler, "crontab"))
     ++	if (!strcmp(scheduler, "launchctl"))
     ++		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
     ++	else if (!strcmp(scheduler, "crontab"))
     + 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
     + 	else
     + 		die("unknown background scheduler: %s", scheduler);
      
       ## t/t7900-maintenance.sh ##
     -@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '
     - 	test_cmp before actual
     - '
     - 
     --test_expect_success 'start from empty cron table' '
     -+test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     - 
     - 	# start registers the repo
     -@@ t/t7900-maintenance.sh: test_expect_success 'start from empty cron table' '
     - 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
     - '
     - 
     --test_expect_success 'stop from existing schedule' '
     -+test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
     +@@ t/t7900-maintenance.sh: test_description='git maintenance builtin'
     + GIT_TEST_COMMIT_GRAPH=0
     + GIT_TEST_MULTI_PACK_INDEX=0
       
     - 	# stop does not unregister the repo
     -@@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
     - 	test_must_be_empty cron.txt
     - '
     - 
     --test_expect_success 'start preserves existing schedule' '
     -+test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
     - 	echo "Important information!" >cron.txt &&
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     ++test_lazy_prereq XMLLINT '
     ++	xmllint --version
     ++'
     ++
     ++test_xmllint () {
     ++	if test_have_prereq XMLLINT
     ++	then
     ++		xmllint --noout "$@"
     ++	else
     ++		true
     ++	fi
     ++}
     ++
     + test_expect_success 'help text' '
     + 	test_expect_code 129 git maintenance -h 2>err &&
     + 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
     +@@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule' '
       	grep "Important information!" cron.txt
       '
       
     -+test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
     -+	write_script print-args "#!/bin/sh\necho \$* >>args" &&
     ++test_expect_success !MINGW 'start and stop macOS maintenance' '
     ++	uid=$(id -u) &&
     ++
     ++	write_script print-args <<-\EOF &&
     ++	echo $* >>args
     ++	EOF
      +
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="./print-args" git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
      +
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	# ~/Library/LaunchAgents
      +	ls "$HOME/Library/LaunchAgents" >actual &&
      +	cat >expect <<-\EOF &&
      +	org.git-scm.git.daily.plist
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
      +	for frequency in hourly daily weekly
      +	do
      +		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
     -+		xmllint --noout "$PLIST" &&
     ++		test_xmllint "$PLIST" &&
      +		grep schedule=$frequency "$PLIST" &&
     -+		echo "bootout gui/$UID $PLIST" >>expect &&
     -+		echo "bootstrap gui/$UID $PLIST" >>expect || return 1
     ++		echo "bootout gui/$uid $PLIST" >>expect &&
     ++		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
      +	done &&
      +	test_cmp expect args &&
      +
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="./print-args" git maintenance stop &&
     ++	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
      +
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	printf "bootout gui/$UID $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
     ++	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
      +		hourly daily weekly >expect &&
      +	test_cmp expect args &&
      +	ls "$HOME/Library/LaunchAgents" >actual &&
     @@ t/t7900-maintenance.sh: test_expect_success 'stop from existing schedule' '
       test_expect_success 'register preserves existing strategy' '
       	git config maintenance.strategy none &&
       	git maintenance register &&
     -
     - ## t/test-lib.sh ##
     -@@ t/test-lib.sh: test_lazy_prereq REBASE_P '
     - 	test -z "$GIT_TEST_SKIP_REBASE_P"
     - '
     - 
     -+test_lazy_prereq MACOS_MAINTENANCE '
     -+	launchctl list
     -+'
     -+
     - # Ensure that no test accidentally triggers a Git command
     - # that runs 'crontab', affecting a user's cron schedule.
     - # Tests that verify the cron integration must set this locally
 4:  ed7a61978f ! 4:  b8d86fb983 maintenance: use Windows scheduled tasks
     @@ Commit message
          logged in, and more fields are populated with the current username and
          SID at run-time by 'schtasks'.
      
     +    Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
     +    specify 'schtasks' as the scheduler, we can test the Windows-specific
     +    logic on a macOS platform. Thus, add a check that the XML file written
     +    by Git is valid when xmllint exists on the system.
     +
          There is a deficiency in the current design. Windows has two kinds of
          applications: GUI applications that start by "winmain()" and console
          applications that start by "main()". Console applications are attached
     @@ Commit message
          short term. In the long term, we can consider creating this GUI
          shim application within core Git, perhaps in contrib/.
      
     -    Helped-by: Eric Sunshine <sunshine@sunshineco.com>
     +    Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
     +    Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     @@ Documentation/git-maintenance.txt: To create more advanced customizations to you
       Part of the linkgit:git[1] suite
      
       ## builtin/gc.c ##
     -@@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
     - 	else
     - 		return remove_plists();
     +@@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
     + 		return launchctl_remove_plists(cmd);
       }
     -+
     -+#elif defined(GIT_WINDOWS_NATIVE)
     -+
     -+static const char *get_frequency(enum schedule_priority schedule)
     -+{
     -+	switch (schedule) {
     -+	case SCHEDULE_HOURLY:
     -+		return "hourly";
     -+	case SCHEDULE_DAILY:
     -+		return "daily";
     -+	case SCHEDULE_WEEKLY:
     -+		return "weekly";
     -+	default:
     -+		BUG("invalid schedule %d", schedule);
     -+	}
     -+}
     -+
     -+static char *get_task_name(const char *frequency)
     + 
     ++static char *schtasks_task_name(const char *frequency)
      +{
      +	struct strbuf label = STRBUF_INIT;
      +	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
      +	return strbuf_detach(&label, NULL);
      +}
      +
     -+static int remove_task(enum schedule_priority schedule)
     ++static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
      +{
      +	int result;
      +	struct strvec args = STRVEC_INIT;
      +	const char *frequency = get_frequency(schedule);
     -+	char *name = get_task_name(frequency);
     -+	const char *schtasks = getenv("GIT_TEST_CRONTAB");
     -+	if (!schtasks)
     -+		schtasks = "schtasks";
     ++	char *name = schtasks_task_name(frequency);
      +
     -+	strvec_split(&args, schtasks);
     ++	strvec_split(&args, cmd);
      +	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
      +
      +	result = run_command_v_opt(args.v, 0);
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +	return result;
      +}
      +
     -+static int remove_scheduled_tasks(void)
     ++static int schtasks_remove_tasks(const char *cmd)
      +{
     -+	return remove_task(SCHEDULE_HOURLY) ||
     -+		remove_task(SCHEDULE_DAILY) ||
     -+		remove_task(SCHEDULE_WEEKLY);
     ++	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
     ++		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
     ++		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
      +}
      +
     -+static int schedule_task(const char *exec_path, enum schedule_priority schedule)
     ++static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
      +{
      +	int result;
      +	struct child_process child = CHILD_PROCESS_INIT;
     -+	const char *xml, *schtasks;
     -+	char *xmlpath, *tempDir;
     -+	FILE *xmlfp;
     ++	const char *xml;
     ++	char *xmlpath;
     ++	struct tempfile *tfile;
      +	const char *frequency = get_frequency(schedule);
     -+	char *name = get_task_name(frequency);
     ++	char *name = schtasks_task_name(frequency);
      +
     -+	tempDir = xstrfmt("%s/temp", the_repository->objects->odb->path);
     -+	xmlpath =  xstrfmt("%s/schedule-%s.xml", tempDir, frequency);
     -+	safe_create_leading_directories(xmlpath);
     -+	xmlfp = xfopen(xmlpath, "w");
     ++	xmlpath =  xstrfmt("%s/schedule-%s.xml",
     ++			   the_repository->objects->odb->path,
     ++			   frequency);
     ++	tfile = create_tempfile(xmlpath);
     ++	if (!tfile || !fdopen_tempfile(tfile, "w"))
     ++		die(_("failed to create '%s'"), xmlpath);
      +
     -+	xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n"
     ++	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
      +	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
      +	      "<Triggers>\n"
      +	      "<CalendarTrigger>\n";
     -+	fprintf(xmlfp, xml);
     ++	fputs(xml, tfile->fp);
      +
      +	switch (schedule) {
      +	case SCHEDULE_HOURLY:
     -+		fprintf(xmlfp,
     ++		fprintf(tfile->fp,
      +			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
      +			"<Enabled>true</Enabled>\n"
      +			"<ScheduleByDay>\n"
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +		break;
      +
      +	case SCHEDULE_DAILY:
     -+		fprintf(xmlfp,
     ++		fprintf(tfile->fp,
      +			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
      +			"<Enabled>true</Enabled>\n"
      +			"<ScheduleByWeek>\n"
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +		break;
      +
      +	case SCHEDULE_WEEKLY:
     -+		fprintf(xmlfp,
     ++		fprintf(tfile->fp,
      +			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
      +			"<Enabled>true</Enabled>\n"
      +			"<ScheduleByWeek>\n"
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +		break;
      +	}
      +
     -+	xml=  "</CalendarTrigger>\n"
     ++	xml = "</CalendarTrigger>\n"
      +	      "</Triggers>\n"
      +	      "<Principals>\n"
      +	      "<Principal id=\"Author\">\n"
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +	      "</Exec>\n"
      +	      "</Actions>\n"
      +	      "</Task>\n";
     -+	fprintf(xmlfp, xml, exec_path, exec_path, frequency);
     -+	fclose(xmlfp);
     -+
     -+	schtasks = getenv("GIT_TEST_CRONTAB");
     -+	if (!schtasks)
     -+		schtasks = "schtasks";
     -+	strvec_split(&child.args, schtasks);
     ++	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
     ++	strvec_split(&child.args, cmd);
      +	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
     ++	close_tempfile_gently(tfile);
      +
      +	child.no_stdout = 1;
      +	child.no_stderr = 1;
     @@ builtin/gc.c: static int platform_update_schedule(int run_maintenance, int fd)
      +		die(_("failed to start schtasks"));
      +	result = finish_command(&child);
      +
     -+	unlink(xmlpath);
     -+	rmdir(tempDir);
     ++	delete_tempfile(&tfile);
      +	free(xmlpath);
      +	free(name);
      +	return result;
      +}
      +
     -+static int add_scheduled_tasks(void)
     ++static int schtasks_schedule_tasks(const char *cmd)
      +{
      +	const char *exec_path = git_exec_path();
      +
     -+	return schedule_task(exec_path, SCHEDULE_HOURLY) ||
     -+		schedule_task(exec_path, SCHEDULE_DAILY) ||
     -+		schedule_task(exec_path, SCHEDULE_WEEKLY);
     ++	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
     ++		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
     ++		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
      +}
      +
     -+static int platform_update_schedule(int run_maintenance, int fd)
     ++static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
      +{
      +	if (run_maintenance)
     -+		return add_scheduled_tasks();
     ++		return schtasks_schedule_tasks(cmd);
      +	else
     -+		return remove_scheduled_tasks();
     ++		return schtasks_remove_tasks(cmd);
      +}
      +
     - #else
       #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
       #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
     -
     - ## t/t7900-maintenance.sh ##
     -@@ t/t7900-maintenance.sh: test_expect_success 'register and unregister' '
     - 	test_cmp before actual
     - '
     - 
     --test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
     -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start from empty cron table' '
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     - 
     - 	# start registers the repo
     -@@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'start from empty cron table' '
     - 	grep "for-each-repo --config=maintenance.repo maintenance run --schedule=weekly" cron.txt
     - '
       
     --test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
     -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'stop from existing schedule' '
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
     +@@ builtin/gc.c: static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
       
     - 	# stop does not unregister the repo
     -@@ t/t7900-maintenance.sh: test_expect_success !MACOS_MAINTENANCE 'stop from existing schedule' '
     - 	test_must_be_empty cron.txt
     - '
     + #if defined(__APPLE__)
     + static const char platform_scheduler[] = "launchctl";
     ++#elif defined(GIT_WINDOWS_NATIVE)
     ++static const char platform_scheduler[] = "schtasks";
     + #else
     + static const char platform_scheduler[] = "crontab";
     + #endif
     +@@ builtin/gc.c: static int update_background_schedule(int enable)
       
     --test_expect_success !MACOS_MAINTENANCE 'start preserves existing schedule' '
     -+test_expect_success !MACOS_MAINTENANCE,!MINGW 'start preserves existing schedule' '
     - 	echo "Important information!" >cron.txt &&
     - 	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
     - 	grep "Important information!" cron.txt
     -@@ t/t7900-maintenance.sh: test_expect_success MACOS_MAINTENANCE 'start and stop macOS maintenance' '
     + 	if (!strcmp(scheduler, "launchctl"))
     + 		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
     ++	else if (!strcmp(scheduler, "schtasks"))
     ++		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
     + 	else if (!strcmp(scheduler, "crontab"))
     + 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
     + 	else
     +
     + ## t/t7900-maintenance.sh ##
     +@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS maintenance' '
       	test_line_count = 0 actual
       '
       
     -+test_expect_success MINGW 'start and stop Windows maintenance' '
     ++test_expect_success 'start and stop Windows maintenance' '
      +	write_script print-args <<-\EOF &&
      +	echo $* >>args
     ++	while test $# -gt 0
     ++	do
     ++		case "$1" in
     ++		/xml) shift; xmlfile=$1; break ;;
     ++		*) shift ;;
     ++		esac
     ++	done
     ++	test -z "$xmlfile" || cp "$xmlfile" .
      +	EOF
      +
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
      +
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/temp/schedule-%s.xml\n" \
     ++	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
      +		hourly hourly daily daily weekly weekly >expect &&
      +	test_cmp expect args &&
      +
     ++	for frequency in hourly daily weekly
     ++	do
     ++		test_xmllint "schedule-$frequency.xml"
     ++	done &&
     ++
      +	rm -f args &&
     -+	GIT_TEST_CRONTAB="/bin/sh print-args" git maintenance stop &&
     ++	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
      +
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&

-- 
gitgitgadget

[PATCH v4 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

As we add this generality, rename GIT_TEST_CRONTAB to
GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as
"<scheduler>:<command>" so we can test platform-specific scheduling
logic even when not on the correct platform. By specifying the
<scheduler> in this string, we will be able to test all three sets of
Git logic from a Linux machine.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 70 ++++++++++++++++++++++++++----------------
 t/t7900-maintenance.sh |  8 ++---
 t/test-lib.sh          |  7 +++--
 3 files changed, 51 insertions(+), 34 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a1..18ae7f7138 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
 	FILE *cron_list, *cron_in;
-	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
-
-	crontab_name = getenv("GIT_TEST_CRONTAB");
-	if (!crontab_name)
-		crontab_name = "crontab";
-
-	strvec_split(&crontab_list.args, crontab_name);
+	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
-	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_list))
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 
 	/* Ignore exit code, as an empty crontab will return error. */
 	finish_command(&crontab_list);
@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
-	strvec_split(&crontab_edit.args, crontab_name);
+	strvec_split(&crontab_edit.args, cmd);
 	crontab_edit.in = -1;
 	crontab_edit.git_cmd = 0;
 
-	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_edit))
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 
 	cron_in = fdopen(crontab_edit.in, "w");
 	if (!cron_in) {
@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static const char platform_scheduler[] = "crontab";
+
+static int update_background_schedule(int enable)
+{
+	int result;
+	const char *scheduler = platform_scheduler;
+	const char *cmd = scheduler;
+	char *testing;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	if (testing) {
+		char *sep = strchr(testing, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+		scheduler = testing;
+		cmd = sep + 1;
 	}
-	fclose(cron_list);
 
-cleanup:
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	if (!strcmp(scheduler, "crontab"))
+		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
+	else
+		die("unknown background scheduler: %s", scheduler);
+
 	rollback_lock_file(&lk);
+	free(testing);
 	return result;
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1..eeb939168d 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '
 '
 
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '
 '
 
 test_expect_success 'stop from existing schedule' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
 	# Operation is idempotent
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 	test_must_be_empty cron.txt
 '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..ddbeee1f5e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '
 '
 
 # Ensure that no test accidentally triggers a Git command
-# that runs 'crontab', affecting a user's cron schedule.
-# Tests that verify the cron integration must set this locally
+# that runs the actual maintenance scheduler, affecting a user's
+# system permanently.
+# Tests that verify the scheduler integration must set this locally
 # to avoid errors.
-GIT_TEST_CRONTAB="exit 1"
+GIT_TEST_MAINT_SCHEDULER="none:exit 1"
-- 
gitgitgadget

[PATCH v4 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc..4c7aac877d 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is `cron`. This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `<path>` string is loaded to specifically use the location for the
+`git` executable used in the `git maintenance start` command. This allows
+for multiple versions to be compatible. However, if the same user runs
+`git maintenance start` with multiple Git executables, then only the
+latest executable will be used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config located at `~/.gitconfig`.
+The `git maintenance` process then determines which maintenance tasks
+are configured to run on each repository with each `<frequency>` using
+the `maintenance.<task>.schedule` config options. These values are loaded
+from the global or repository config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
+for advanced scheduling techniques. Please do use the full path and
+`--exec-path` techniques from the default schedule to ensure you are
+executing the correct binaries in your schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

Re: [PATCH v4 2/4] maintenance: include 'cron' details in docs

[Eric Sunshine]

On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Advanced and expert users may want to know how 'git maintenance start'
> schedules background maintenance in order to customize their own
> schedules beyond what the maintenance.* config values allow. Start a new
> set of sections in git-maintenance.txt that describe how 'cron' is used
> to run these tasks.
>
> This is particularly valuable for users who want to inspect what Git is
> doing or for users who want to customize the schedule further. Having a
> baseline can provide a way forward for users who have never worked with
> cron schedules.
>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
> +The comments are used as a region to mark the schedule as written by Git.
> +Any modifications within this region will be completely deleted by
> +`git maintenance stop` or overwritten by `git maintenance start`.
> +
> +The `<path>` string is loaded to specifically use the location for the
> +`git` executable used in the `git maintenance start` command. This allows
> +for multiple versions to be compatible. However, if the same user runs
> +`git maintenance start` with multiple Git executables, then only the
> +latest executable will be used.

It looks like this section in v4 got accidentally reverted to the
wording from v2, whereas v3 had been changed to:

    The `crontab` entry specifies the full path of the `git`
    executable to ensure that the executed `git` command is the same
    one with which `git maintenance start` was issued independent of
    `PATH`. If the same user runs `git maintenance start` with
    multiple Git executables, then only the latest executable is used.

> +These commands use `git for-each-repo --config=maintenance.repo` to run
> +`git maintenance run --schedule=<frequency>` on each repository listed in
> +the multi-valued `maintenance.repo` config option. These are typically
> +loaded from the user-specific global config located at `~/.gitconfig`.
> +The `git maintenance` process then determines which maintenance tasks
> +are configured to run on each repository with each `<frequency>` using
> +the `maintenance.<task>.schedule` config options. These values are loaded
> +from the global or repository config values.

Same problem here. This wording is from v2, whereas v3 had been
changed to say generically "user-specific global config" rather than
mentioning `~/.gitconfig` explicitly.

> +If the config values are insufficient to achieve your desired background
> +maintenance schedule, then you can create your own schedule. If you run
> +`crontab -e`, then an editor will load with your user-specific `cron`
> +schedule. In that editor, you can add your own schedule lines. You could
> +start by adapting the default schedule listed earlier, or you could read
> +https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
> +for advanced scheduling techniques. Please do use the full path and
> +`--exec-path` techniques from the default schedule to ensure you are
> +executing the correct binaries in your schedule.

And here too. v3 had updated this to say only "crontab(5)" rather than
providing an explicit URL.

Re: [PATCH v4 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee]

On 11/17/2020 7:34 PM, Eric Sunshine wrote:
> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> Advanced and expert users may want to know how 'git maintenance start'
>> schedules background maintenance in order to customize their own
>> schedules beyond what the maintenance.* config values allow. Start a new
>> set of sections in git-maintenance.txt that describe how 'cron' is used
>> to run these tasks.
>>
>> This is particularly valuable for users who want to inspect what Git is
>> doing or for users who want to customize the schedule further. Having a
>> baseline can provide a way forward for users who have never worked with
>> cron schedules.
>>
>> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
>> ---
>> diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
>> @@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
>> +The comments are used as a region to mark the schedule as written by Git.
>> +Any modifications within this region will be completely deleted by
>> +`git maintenance stop` or overwritten by `git maintenance start`.
>> +
>> +The `<path>` string is loaded to specifically use the location for the
>> +`git` executable used in the `git maintenance start` command. This allows
>> +for multiple versions to be compatible. However, if the same user runs
>> +`git maintenance start` with multiple Git executables, then only the
>> +latest executable will be used.
> 
> It looks like this section in v4 got accidentally reverted to the
> wording from v2, whereas v3 had been changed to:> 
>     The `crontab` entry specifies the full path of the `git`
>     executable to ensure that the executed `git` command is the same
>     one with which `git maintenance start` was issued independent of
>     `PATH`. If the same user runs `git maintenance start` with
>     multiple Git executables, then only the latest executable is used.

Embarassing. Thanks for the catch.

Thanks,
-Stolee

[PATCH v4 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2]. The current design does not preclude a future version that
detects the available fatures of 'launchctl' to use the older
commands. However, it is best to rely on the newest version since
Apple might completely remove the deprecated version on short
notice.

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

The need for a user id requires us to run 'id -u' which works on
POSIX systems but not Windows. The test therefore has a prerequisite
that we are not on Windows. The cross-platform logic still allows us to
test the macOS logic on a Linux machine.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_MAINT_SCHEDULER.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format. This is useful for any system that might contain
the tool, so use it whenever it is available.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  40 +++++++
 builtin/gc.c                      | 188 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  58 +++++++++
 3 files changed, 285 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 4c7aac877d..f2d59f2bcc 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,46 @@ for advanced scheduling techniques. Please do use the full path and
 executing the correct binaries in your schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process does not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to schedule timed jobs in macOS. Scheduling
+maintenance through `git maintenance (start|stop)` requires some
+`launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/org.git-scm.git*
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+launchctl.plist(5) for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 18ae7f7138..782769f243 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *launchctl_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *launchctl_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static char *launchctl_get_uid(void)
+{
+	return xstrfmt("gui/%d", getuid());
+}
+
+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	char *uid = launchctl_get_uid();
+
+	strvec_split(&child.args, cmd);
+	if (enable)
+		strvec_push(&child.args, "bootstrap");
+	else
+		strvec_push(&child.args, "bootout");
+	strvec_push(&child.args, uid);
+	strvec_push(&child.args, filename);
+
+	child.no_stderr = 1;
+	child.no_stdout = 1;
+
+	if (start_command(&child))
+		die(_("failed to start launchctl"));
+
+	result = finish_command(&child);
+
+	free(uid);
+	return result;
+}
+
+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+	int result = launchctl_boot_plist(0, filename, cmd);
+	unlink(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int launchctl_remove_plists(const char *cmd)
+{
+	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = xfopen(filename, "w");
+
+	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+	fclose(plist);
+
+	/* bootout might fail if not already running, so ignore */
+	launchctl_boot_plist(0, filename, cmd);
+	if (launchctl_boot_plist(1, filename, cmd))
+		die(_("failed to bootstrap service %s"), filename);
+
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int launchctl_add_plists(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return launchctl_add_plists(cmd);
+	else
+		return launchctl_remove_plists(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+#if defined(__APPLE__)
+static const char platform_scheduler[] = "launchctl";
+#else
 static const char platform_scheduler[] = "crontab";
+#endif
 
 static int update_background_schedule(int enable)
 {
@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "crontab"))
+	if (!strcmp(scheduler, "launchctl"))
+		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
 		die("unknown background scheduler: %s", scheduler);
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index eeb939168d..6d37312901 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -7,6 +7,19 @@ test_description='git maintenance builtin'
 GIT_TEST_COMMIT_GRAPH=0
 GIT_TEST_MULTI_PACK_INDEX=0
 
+test_lazy_prereq XMLLINT '
+	xmllint --version
+'
+
+test_xmllint () {
+	if test_have_prereq XMLLINT
+	then
+		xmllint --noout "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success !MINGW 'start and stop macOS maintenance' '
+	uid=$(id -u) &&
+
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		test_xmllint "$PLIST" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$uid $PLIST" >>expect &&
+		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args &&
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	test_line_count = 0 actual
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v4 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> +static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
> +{
> +       child.no_stderr = 1;
> +       child.no_stdout = 1;
> +       if (start_command(&child))
> +               die(_("failed to start launchctl"));

Did you have any thoughts on the observation I made in a followup
response[1] during review of v3 in which I suggested that we might be
able to avoid suppressing stderr (and stdout) here? In particular, the
idea was that if, in launchctl_schedule_plist(), we do a simple
existence check for the .plist file and only call
launchctl_boot_plist(0,...) to `bootout` the .plist file if it exists,
then we shouldn't need to muck with stderr/stdout suppression. The
benefit is that if `bootout` fails for some reason, then the user
would see the (hopefully) meaningful error message emitted `launchctl
bootout`.

The same .plist existence check could be done in
launchctl_remove_plist() before trying to `bootout` the file.

Anyhow, such refinement can be done later is desired, so not worth a
re-roll, but I was curious about your thoughts on the issue.

[1]: https://lore.kernel.org/git/CAPig+cTRJb-fn2R6rJO1hkeCc_ehVhkNufO4=LhtPQudVeonnA@mail.gmail.com/

Re: [PATCH v4 3/4] maintenance: use launchctl on macOS

[Derrick Stolee]

On 11/18/2020 1:45 AM, Eric Sunshine wrote:
> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> +static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
>> +{
>> +       child.no_stderr = 1;
>> +       child.no_stdout = 1;
>> +       if (start_command(&child))
>> +               die(_("failed to start launchctl"));
> 
> Did you have any thoughts on the observation I made in a followup
> response[1] during review of v3 in which I suggested that we might be
> able to avoid suppressing stderr (and stdout) here? In particular, the
> idea was that if, in launchctl_schedule_plist(), we do a simple
> existence check for the .plist file and only call
> launchctl_boot_plist(0,...) to `bootout` the .plist file if it exists,
> then we shouldn't need to muck with stderr/stdout suppression. The
> benefit is that if `bootout` fails for some reason, then the user
> would see the (hopefully) meaningful error message emitted `launchctl
> bootout`.
> 
> The same .plist existence check could be done in
> launchctl_remove_plist() before trying to `bootout` the file.

If the file exists but isn't still registered with launchd, then
the bootout command will send output

"<path>.plist: Could not find specified service"

and return a failure code. This output isn't helpful to users,
since we still are in the desired state afterwards.

> Anyhow, such refinement can be done later is desired, so not worth a
> re-roll, but I was curious about your thoughts on the issue.

This pattern of squashing the output even in the successful case
is important for Windows where schtasks sends a line of output for
each task being registered!

I think it would be a reasonable extension to store the error
message for logging or communicating to the user if we actually
need it, but I don't believe we should be piping the output
directly.

Thanks,
-Stolee

[PATCH v4 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
specify 'schtasks' as the scheduler, we can test the Windows-specific
logic on a macOS platform. Thus, add a check that the XML file written
by Git is valid when xmllint exists on the system.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 165 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  40 ++++++++
 3 files changed, 227 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index f2d59f2bcc..e1adfff6db 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see
 launchctl.plist(5) for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 782769f243..f6c42f96c1 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
 		return launchctl_remove_plists(cmd);
 }
 
+static char *schtasks_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	strvec_split(&args, cmd);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int schtasks_remove_tasks(const char *cmd)
+{
+	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
+		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *xml;
+	char *xmlpath;
+	struct tempfile *tfile;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	xmlpath =  xstrfmt("%s/schedule-%s.xml",
+			   the_repository->objects->odb->path,
+			   frequency);
+	tfile = create_tempfile(xmlpath);
+	if (!tfile || !fdopen_tempfile(tfile, "w"))
+		die(_("failed to create '%s'"), xmlpath);
+
+	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fputs(xml, tfile->fp);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml = "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
+	close_tempfile_gently(tfile);
+
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+
+	if (start_command(&child))
+		die(_("failed to start schtasks"));
+	result = finish_command(&child);
+
+	delete_tempfile(&tfile);
+	free(xmlpath);
+	free(name);
+	return result;
+}
+
+static int schtasks_schedule_tasks(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return schtasks_schedule_tasks(cmd);
+	else
+		return schtasks_remove_tasks(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1761,6 +1922,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 
 #if defined(__APPLE__)
 static const char platform_scheduler[] = "launchctl";
+#elif defined(GIT_WINDOWS_NATIVE)
+static const char platform_scheduler[] = "schtasks";
 #else
 static const char platform_scheduler[] = "crontab";
 #endif
@@ -1789,6 +1952,8 @@ static int update_background_schedule(int enable)
 
 	if (!strcmp(scheduler, "launchctl"))
 		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "schtasks"))
+		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
 	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 6d37312901..a26ff22541 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -453,6 +453,46 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '
 	test_line_count = 0 actual
 '
 
+test_expect_success 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	while test $# -gt 0
+	do
+		case "$1" in
+		/xml) shift; xmlfile=$1; break ;;
+		*) shift ;;
+		esac
+	done
+	test -z "$xmlfile" || cp "$xmlfile" .
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
+		hourly hourly daily daily weekly weekly >expect &&
+	test_cmp expect args &&
+
+	for frequency in hourly daily weekly
+	do
+		test_xmllint "schedule-$frequency.xml"
+	done &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> [...]
> Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
> specify 'schtasks' as the scheduler, we can test the Windows-specific
> logic on a macOS platform. Thus, add a check that the XML file written
> by Git is valid when xmllint exists on the system.

Nit: xmllint can be installed on Linux (and likely other platforms),
as well, so it's not clear why this calls out macOS specially. More
generally, it may not be important to call out xmllint at all in the
commit message; it's just _one_ thing being checked by a test which is
checking several other things not called out individually by the
commit message. Anyhow, this is minor; not worth a re-roll.

> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +{
> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
> +                          the_repository->objects->odb->path,
> +                          frequency);

I missed this in the earlier rounds since I wasn't paying close enough
attention, but placing this XML file within the object database
directory (.git/objects/) feels rather odd, even if it is just a
temporary file. Using the .git/ directory itself might be better,
perhaps like this:

    struct strbuf xmlpath = STRBUF_INIT;
    strbuf_git_common_path(&xmlpath, the_repository,
        "schtasks-%s.xml", frequency);
    ...
    strbuf_release(&xmlpath);

Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/18/2020 2:15 AM, Eric Sunshine wrote:
> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> [...]
>> Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
>> specify 'schtasks' as the scheduler, we can test the Windows-specific
>> logic on a macOS platform. Thus, add a check that the XML file written
>> by Git is valid when xmllint exists on the system.
> 
> Nit: xmllint can be installed on Linux (and likely other platforms),
> as well, so it's not clear why this calls out macOS specially. More
> generally, it may not be important to call out xmllint at all in the
> commit message; it's just _one_ thing being checked by a test which is
> checking several other things not called out individually by the
> commit message. Anyhow, this is minor; not worth a re-roll.

Sorry, it should just say "other platforms"

>> diff --git a/builtin/gc.c b/builtin/gc.c
>> @@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
>> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
>> +{
>> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
>> +                          the_repository->objects->odb->path,
>> +                          frequency);
> 
> I missed this in the earlier rounds since I wasn't paying close enough
> attention, but placing this XML file within the object database
> directory (.git/objects/) feels rather odd, even if it is just a
> temporary file. Using the .git/ directory itself might be better,
> perhaps like this:
> 
>     struct strbuf xmlpath = STRBUF_INIT;
>     strbuf_git_common_path(&xmlpath, the_repository,
>         "schtasks-%s.xml", frequency);
>     ...
>     strbuf_release(&xmlpath);

It does look odd, and in this case we could use the .git directory
instead. I specifically use the objects directory for the maintenance
lock in 'git maintenance run' to allow maintenance to run when
GIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain
object databases that don't have a full .git directory around them.

Thanks,
-Stolee

Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Wed, Nov 18, 2020 at 1:30 PM Derrick Stolee <stolee@gmail.com> wrote:
> On 11/18/2020 2:15 AM, Eric Sunshine wrote:
> > On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
> > <gitgitgadget@gmail.com> wrote:
> >> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
> >> +                          the_repository->objects->odb->path,
> >> +                          frequency);
> >
> > I missed this in the earlier rounds since I wasn't paying close enough
> > attention, but placing this XML file within the object database
> > directory (.git/objects/) feels rather odd, even if it is just a
> > temporary file. Using the .git/ directory itself might be better,
> > perhaps like this:
>
> It does look odd, and in this case we could use the .git directory
> instead. I specifically use the objects directory for the maintenance
> lock in 'git maintenance run' to allow maintenance to run when
> GIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain
> object databases that don't have a full .git directory around them.

I guess I'm confused. Won't a Git "common" directory exist even for
such a case when GIT_OBJECT_DIRECTORY is pointing elsewhere, whether
the "common" directory is .git/ or a bare repository, or whatnot?

Anyhow, this brings us back to my original suggestion of creating
these temporary files in a genuine temporary directory (/tmp or
$TMPDIR or $TEMP) instead of arbitrarily choosing a path within the
repository itself. An important reason for using a genuine temporary
directory for these temporary XML files is that it makes it less
confusing for those who come along later and try to understand this
code; they won't have to puzzle out why it is using a repository
location for a file which is clearly temporary.

To make this really simple, you could use one of the
x?mks_tempfile_t*() functions from tempfile.h which will automatically
place the file in $TMPDIR, thus relieving this code from having to
make the choice. Doing so would simplify this code even further since
you would replace create_tempfile() with x?mks_tempfile_t*(), and
wouldn't have to maintain (or free) `xmlpath` manually.

As for the test script, the `print-args` is already picking up the
pathname of the temporary file specified by the /xml option, so it
should be possible to make the rest of the test work with the
generated temporary filename.

Re: [PATCH v4 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee]

On 11/18/2020 3:54 PM, Eric Sunshine wrote:
> On Wed, Nov 18, 2020 at 1:30 PM Derrick Stolee <stolee@gmail.com> wrote:
>> On 11/18/2020 2:15 AM, Eric Sunshine wrote:
>>> On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
>>> <gitgitgadget@gmail.com> wrote:
>>>> +       xmlpath =  xstrfmt("%s/schedule-%s.xml",
>>>> +                          the_repository->objects->odb->path,
>>>> +                          frequency);
>>>
>>> I missed this in the earlier rounds since I wasn't paying close enough
>>> attention, but placing this XML file within the object database
>>> directory (.git/objects/) feels rather odd, even if it is just a
>>> temporary file. Using the .git/ directory itself might be better,
>>> perhaps like this:
>>
>> It does look odd, and in this case we could use the .git directory
>> instead. I specifically use the objects directory for the maintenance
>> lock in 'git maintenance run' to allow maintenance to run when
>> GIT_OBJECT_DIRECTORY points to an alternate, allowing us to maintain
>> object databases that don't have a full .git directory around them.
> 
> I guess I'm confused. Won't a Git "common" directory exist even for
> such a case when GIT_OBJECT_DIRECTORY is pointing elsewhere, whether
> the "common" directory is .git/ or a bare repository, or whatnot?

The reason to use the object dir for the 'git maintenance run' lock
is to avoid multiple enlistments pointing at a common alternate from
running concurrent maintenance on the same object directory.

That doesn't really apply to the temp files in this patch.

> Anyhow, this brings us back to my original suggestion of creating
> these temporary files in a genuine temporary directory (/tmp or
> $TMPDIR or $TEMP) instead of arbitrarily choosing a path within the
> repository itself. An important reason for using a genuine temporary
> directory for these temporary XML files is that it makes it less
> confusing for those who come along later and try to understand this
> code; they won't have to puzzle out why it is using a repository
> location for a file which is clearly temporary.
> 
> To make this really simple, you could use one of the
> x?mks_tempfile_t*() functions from tempfile.h which will automatically
> place the file in $TMPDIR, thus relieving this code from having to
> make the choice. Doing so would simplify this code even further since
> you would replace create_tempfile() with x?mks_tempfile_t*(), and
> wouldn't have to maintain (or free) `xmlpath` manually.
> 
> As for the test script, the `print-args` is already picking up the
> pathname of the temporary file specified by the /xml option, so it
> should be possible to make the rest of the test work with the
> generated temporary filename.
 
I'll adopt your recommendations here. Thanks.
-Stolee

Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Tue, Nov 17, 2020 at 09:13:14PM +0000, Derrick Stolee via GitGitGadget wrote:
> Updates in V4
> =============
>  * Eric did an excellent job providing a patch that cleans up several parts
>    of my series. The most impressive is his mechanism for testing the
>    platform-specific Git logic in a way that is (mostly) platform-agnostic.
>    
>  * Windows doesn't have the 'id' command, so we cannot run the macOS
>    platform test on Windows.

This is easy to resolve. Drop in the following patch and then replace
the `$(id -u)` invocation in the test with `$(test-tool getuid)`.
This way, the test should work on any platform since both
launchctl_get_uid() and `test-tool` will retrieve identical values for
UID.

--- >8 ---
From 84f623bcaec156082c0e7151f40aef18575e6f86 Mon Sep 17 00:00:00 2001
From: Eric Sunshine <sunshine@sunshineco.com>
Date: Tue, 17 Nov 2020 18:30:10 -0500
Subject: [PATCH] test-helper: add `getuid` subcommand

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 Makefile               | 1 +
 t/helper/test-getuid.c | 7 +++++++
 t/helper/test-tool.c   | 1 +
 t/helper/test-tool.h   | 1 +
 4 files changed, 10 insertions(+)
 create mode 100644 t/helper/test-getuid.c

diff --git a/Makefile b/Makefile
index 790a883932..230aff5e5c 100644
--- a/Makefile
+++ b/Makefile
@@ -706,6 +706,7 @@ TEST_BUILTINS_OBJS += test-dump-untracked-cache.o
 TEST_BUILTINS_OBJS += test-example-decorate.o
 TEST_BUILTINS_OBJS += test-genrandom.o
 TEST_BUILTINS_OBJS += test-genzeros.o
+TEST_BUILTINS_OBJS += test-getuid.o
 TEST_BUILTINS_OBJS += test-hash-speed.o
 TEST_BUILTINS_OBJS += test-hash.o
 TEST_BUILTINS_OBJS += test-hashmap.o
diff --git a/t/helper/test-getuid.c b/t/helper/test-getuid.c
new file mode 100644
index 0000000000..d741302461
--- /dev/null
+++ b/t/helper/test-getuid.c
@@ -0,0 +1,7 @@
+#include "test-tool.h"
+
+int cmd__getuid(int argc, const char **argv)
+{
+	printf("%d\n", getuid());
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index a0d3966b29..ab206541df 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -30,6 +30,7 @@ static struct test_cmd cmds[] = {
 	{ "example-decorate", cmd__example_decorate },
 	{ "genrandom", cmd__genrandom },
 	{ "genzeros", cmd__genzeros },
+	{ "getuid", cmd__getuid },
 	{ "hashmap", cmd__hashmap },
 	{ "hash-speed", cmd__hash_speed },
 	{ "index-version", cmd__index_version },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index 07034d3f38..caee0a3667 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -20,6 +20,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv);
 int cmd__example_decorate(int argc, const char **argv);
 int cmd__genrandom(int argc, const char **argv);
 int cmd__genzeros(int argc, const char **argv);
+int cmd__getuid(int argc, const char **argv);
 int cmd__hashmap(int argc, const char **argv);
 int cmd__hash_speed(int argc, const char **argv);
 int cmd__index_version(int argc, const char **argv);
-- 
2.29.2.454.gaff20da3a2

Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 11/17/2020 6:36 PM, Eric Sunshine wrote:
> On Tue, Nov 17, 2020 at 09:13:14PM +0000, Derrick Stolee via GitGitGadget wrote:
>> Updates in V4
>> =============
>>  * Eric did an excellent job providing a patch that cleans up several parts
>>    of my series. The most impressive is his mechanism for testing the
>>    platform-specific Git logic in a way that is (mostly) platform-agnostic.
>>    
>>  * Windows doesn't have the 'id' command, so we cannot run the macOS
>>    platform test on Windows.
> 
> This is easy to resolve. Drop in the following patch and then replace
> the `$(id -u)` invocation in the test with `$(test-tool getuid)`.
> This way, the test should work on any platform since both
> launchctl_get_uid() and `test-tool` will retrieve identical values for
> UID.

I was giving your 'test-tool getuid' idea a try, and found that _also_
the $HOME environment variable differs from the format we expect in these
subcommands:

                   $HOME: C:\...
  argument in subcommand: /c/...

So, there is another reason why these tests don't work on Windows. I'm
of the opinion that maybe it's not worth _that_ level of cross-platform
testing.

Unless I'm missing something simple about a $HOME alternative here, this
seems to be more work than the resulting value. Personally, I'm happy
with the benefit you've already provided in allowing Linux to test all
platforms.

Thanks,
-Stolee

Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Mon, Nov 23, 2020 at 9:20 PM Derrick Stolee <stolee@gmail.com> wrote:
> I was giving your 'test-tool getuid' idea a try, and found that _also_
> the $HOME environment variable differs from the format we expect in these
> subcommands:
>
>                    $HOME: C:\...
>   argument in subcommand: /c/...

Where does this problem crop up exactly? Is the test doing a literal
comparison against the value in $HOME?

> So, there is another reason why these tests don't work on Windows. I'm
> of the opinion that maybe it's not worth _that_ level of cross-platform
> testing.
>
> Unless I'm missing something simple about a $HOME alternative here, this
> seems to be more work than the resulting value. Personally, I'm happy
> with the benefit you've already provided in allowing Linux to test all
> platforms.

Indeed, it's not worth investing a lot of additional time into it. And
it's certainly not a good reason to hold up the series. Moreover, this
is the sort of thing which can be refined/handled later if someone
wants to take a shot at it.

Re: [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Tue, Nov 17, 2020 at 4:13 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>  * I noticed far too late that while my example XML files had been edited
>    with UTF-8 encoding, Git is actually writing them as US-ASCII. Somehow
>    xmllint and launchd are not complaining, but schtasks does complain.
>    Unfortunately, I cannot find a way to catch this problem other than to
>    install my tip version on all three platforms and go through the entire
>    git maintenance start process, and double-check that the processes are
>    running on the hour.

I'm having trouble understanding what problem is being described here
and whether or not it has been solved by v4.

I might guess that you are saying that `schtasks` insists upon seeing
a UTF-8 BOM at the start of the XML file since the XML file declares
itself as UTF-8, but that Git is (quite naturally) writing out the
file without the UTF-8 BOM.

> -    xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
> +    xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"

If the above speculation is correct, and if `schtasks` is happy with
the plain text file (lacking UTF-8 BOM) declaring itself as US-ASCII,
then this seems a reasonable solution. And it's easy to test that this
doesn't get broken. After validating the file with `xmllint`, also
grep it for US-ASCII, perhaps like this:

    test_xmllint "schedule-$frequency.xml" &&
    grep "encoding=.US-ASCII." "schedule-$frequency.xml"

[PATCH v5 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.

Updates in V5
=============

 * Fixed docs from PATCH 2 to match those in v3.
   
   
 * Despite my best efforts, I was unable to make the macOS tests work on
   Windows, so they are still marked with "!MINGW". I updated the commit
   message to describe my problems there.
   
   
 * The Windows platform now uses xmks_tempfile() to create the XML files for
   'schtasks'. This led to some test fallout since the pathnames are no
   longer predictable.
   
   

Thanks, -Stolee

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 116 +++++++++
 builtin/gc.c                      | 416 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 110 +++++++-
 t/test-lib.sh                     |   7 +-
 4 files changed, 615 insertions(+), 34 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v4:

 1:  4807342b00 = 1:  4807342b00 maintenance: extract platform-specific scheduling
 2:  99170df462 ! 2:  7cc70a8fe7 maintenance: include 'cron' details in docs
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +---------------------------------------
      +
      +The standard mechanism for scheduling background tasks on POSIX systems
     -+is `cron`. This tool executes commands based on a given schedule. The
     ++is cron(8). This tool executes commands based on a given schedule. The
      +current list of user-scheduled tasks can be found by running `crontab -l`.
      +The schedule written by `git maintenance start` is similar to this:
      +
     @@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
      +Any modifications within this region will be completely deleted by
      +`git maintenance stop` or overwritten by `git maintenance start`.
      +
     -+The `<path>` string is loaded to specifically use the location for the
     -+`git` executable used in the `git maintenance start` command. This allows
     -+for multiple versions to be compatible. However, if the same user runs
     -+`git maintenance start` with multiple Git executables, then only the
     -+latest executable will be used.
     ++The `crontab` entry specifies the full path of the `git` executable to
     ++ensure that the executed `git` command is the same one with which
     ++`git maintenance start` was issued independent of `PATH`. If the same user
     ++runs `git maintenance start` with multiple Git executables, then only the
     ++latest executable is used.
      +
      +These commands use `git for-each-repo --config=maintenance.repo` to run
      +`git maintenance run --schedule=<frequency>` on each repository listed in
      +the multi-valued `maintenance.repo` config option. These are typically
     -+loaded from the user-specific global config located at `~/.gitconfig`.
     -+The `git maintenance` process then determines which maintenance tasks
     -+are configured to run on each repository with each `<frequency>` using
     -+the `maintenance.<task>.schedule` config options. These values are loaded
     -+from the global or repository config values.
     ++loaded from the user-specific global config. The `git maintenance` process
     ++then determines which maintenance tasks are configured to run on each
     ++repository with each `<frequency>` using the `maintenance.<task>.schedule`
     ++config options. These values are loaded from the global or repository
     ++config values.
      +
      +If the config values are insufficient to achieve your desired background
      +maintenance schedule, then you can create your own schedule. If you run
      +`crontab -e`, then an editor will load with your user-specific `cron`
      +schedule. In that editor, you can add your own schedule lines. You could
      +start by adapting the default schedule listed earlier, or you could read
     -+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
     -+for advanced scheduling techniques. Please do use the full path and
     -+`--exec-path` techniques from the default schedule to ensure you are
     -+executing the correct binaries in your schedule.
     ++the crontab(5) documentation for advanced scheduling techniques. Please
     ++do use the full path and `--exec-path` techniques from the default
     ++schedule to ensure you are executing the correct binaries in your
     ++schedule.
      +
       
       GIT
 3:  ed0a0011fb ! 3:  cd015a5cbd maintenance: use launchctl on macOS
     @@ Commit message
          subcommand will succeed, if such a task already exists.
      
          The need for a user id requires us to run 'id -u' which works on
     -    POSIX systems but not Windows. The test therefore has a prerequisite
     -    that we are not on Windows. The cross-platform logic still allows us to
     -    test the macOS logic on a Linux machine.
     +    POSIX systems but not Windows. Further, the need for fully-qualitifed
     +    path names including $HOME behaves differently in the Git internals and
     +    the external test suite. The $HOME variable starts with "C:\..." instead
     +    of the "/c/..." that is provided by Git in these subcommands. The test
     +    therefore has a prerequisite that we are not on Windows. The cross-
     +    platform logic still allows us to test the macOS logic on a Linux
     +    machine.
      
          We can verify the commands that were run by 'git maintenance start'
          and 'git maintenance stop' by injecting a script that writes the
     @@ Commit message
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
      
       ## Documentation/git-maintenance.txt ##
     -@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
     - executing the correct binaries in your schedule.
     +@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your
     + schedule.
       
       
      +BACKGROUND MAINTENANCE ON MACOS SYSTEMS
 4:  b8d86fb983 ! 4:  ac9a28bea3 maintenance: use Windows scheduled tasks
     @@ Commit message
      
          Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
          specify 'schtasks' as the scheduler, we can test the Windows-specific
     -    logic on a macOS platform. Thus, add a check that the XML file written
     +    logic on other platforms. Thus, add a check that the XML file written
          by Git is valid when xmllint exists on the system.
      
     +    Since we use a temporary file for the XML files sent to 'schtasks', we
     +    must copy the file to a predictable filename. Use the number of lines in
     +    the 'args' file to provide a filename for xmllint. Instead of an exact
     +    match on the 'args' file, we 'grep' for the arguments other than the
     +    filename.
     +
          There is a deficiency in the current design. Windows has two kinds of
          applications: GUI applications that start by "winmain()" and console
          applications that start by "main()". Console applications are attached
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	int result;
      +	struct child_process child = CHILD_PROCESS_INIT;
      +	const char *xml;
     -+	char *xmlpath;
      +	struct tempfile *tfile;
      +	const char *frequency = get_frequency(schedule);
      +	char *name = schtasks_task_name(frequency);
      +
     -+	xmlpath =  xstrfmt("%s/schedule-%s.xml",
     -+			   the_repository->objects->odb->path,
     -+			   frequency);
     -+	tfile = create_tempfile(xmlpath);
     ++	tfile = xmks_tempfile("schedule_XXXXXX");
      +	if (!tfile || !fdopen_tempfile(tfile, "w"))
     -+		die(_("failed to create '%s'"), xmlpath);
     ++		die(_("failed to create temp xml file"));
      +
      +	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
      +	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	      "</Task>\n";
      +	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
      +	strvec_split(&child.args, cmd);
     -+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
     ++	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", tfile->filename.buf, NULL);
      +	close_tempfile_gently(tfile);
      +
      +	child.no_stdout = 1;
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	result = finish_command(&child);
      +
      +	delete_tempfile(&tfile);
     -+	free(xmlpath);
      +	free(name);
      +	return result;
      +}
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +		*) shift ;;
      +		esac
      +	done
     -+	test -z "$xmlfile" || cp "$xmlfile" .
     ++	lines=$(wc -l args | awk "{print \$1;}")
     ++	test -z "$xmlfile" || cp "$xmlfile" "schedule-$lines.xml"
      +	EOF
      +
      +	rm -f args &&
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
     -+		hourly hourly daily daily weekly weekly >expect &&
     -+	test_cmp expect args &&
     -+
      +	for frequency in hourly daily weekly
      +	do
     -+		test_xmllint "schedule-$frequency.xml"
     ++		grep "/create /tn Git Maintenance ($frequency) /f /xml" args \
     ++			|| return 1
     ++	done &&
     ++
     ++	for i in 1 2 3
     ++	do
     ++		test_xmllint "schedule-$i.xml" &&
     ++		grep "encoding=.US-ASCII." "schedule-$i.xml" || return 1
      +	done &&
      +
      +	rm -f args &&

-- 
gitgitgadget

[PATCH v5 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

As we add this generality, rename GIT_TEST_CRONTAB to
GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as
"<scheduler>:<command>" so we can test platform-specific scheduling
logic even when not on the correct platform. By specifying the
<scheduler> in this string, we will be able to test all three sets of
Git logic from a Linux machine.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 70 ++++++++++++++++++++++++++----------------
 t/t7900-maintenance.sh |  8 ++---
 t/test-lib.sh          |  7 +++--
 3 files changed, 51 insertions(+), 34 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a1..18ae7f7138 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
 	FILE *cron_list, *cron_in;
-	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
-
-	crontab_name = getenv("GIT_TEST_CRONTAB");
-	if (!crontab_name)
-		crontab_name = "crontab";
-
-	strvec_split(&crontab_list.args, crontab_name);
+	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
-	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_list))
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 
 	/* Ignore exit code, as an empty crontab will return error. */
 	finish_command(&crontab_list);
@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
-	strvec_split(&crontab_edit.args, crontab_name);
+	strvec_split(&crontab_edit.args, cmd);
 	crontab_edit.in = -1;
 	crontab_edit.git_cmd = 0;
 
-	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_edit))
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 
 	cron_in = fdopen(crontab_edit.in, "w");
 	if (!cron_in) {
@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static const char platform_scheduler[] = "crontab";
+
+static int update_background_schedule(int enable)
+{
+	int result;
+	const char *scheduler = platform_scheduler;
+	const char *cmd = scheduler;
+	char *testing;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	if (testing) {
+		char *sep = strchr(testing, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+		scheduler = testing;
+		cmd = sep + 1;
 	}
-	fclose(cron_list);
 
-cleanup:
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	if (!strcmp(scheduler, "crontab"))
+		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
+	else
+		die("unknown background scheduler: %s", scheduler);
+
 	rollback_lock_file(&lk);
+	free(testing);
 	return result;
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1..eeb939168d 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '
 '
 
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '
 '
 
 test_expect_success 'stop from existing schedule' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
 	# Operation is idempotent
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 	test_must_be_empty cron.txt
 '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed76..ddbeee1f5e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '
 '
 
 # Ensure that no test accidentally triggers a Git command
-# that runs 'crontab', affecting a user's cron schedule.
-# Tests that verify the cron integration must set this locally
+# that runs the actual maintenance scheduler, affecting a user's
+# system permanently.
+# Tests that verify the scheduler integration must set this locally
 # to avoid errors.
-GIT_TEST_CRONTAB="exit 1"
+GIT_TEST_MAINT_SCHEDULER="none:exit 1"
-- 
gitgitgadget

[PATCH v5 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc..1aa1112418 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is cron(8). This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `crontab` entry specifies the full path of the `git` executable to
+ensure that the executed `git` command is the same one with which
+`git maintenance start` was issued independent of `PATH`. If the same user
+runs `git maintenance start` with multiple Git executables, then only the
+latest executable is used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config. The `git maintenance` process
+then determines which maintenance tasks are configured to run on each
+repository with each `<frequency>` using the `maintenance.<task>.schedule`
+config options. These values are loaded from the global or repository
+config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+the crontab(5) documentation for advanced scheduling techniques. Please
+do use the full path and `--exec-path` techniques from the default
+schedule to ensure you are executing the correct binaries in your
+schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

[PATCH v5 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2]. The current design does not preclude a future version that
detects the available fatures of 'launchctl' to use the older
commands. However, it is best to rely on the newest version since
Apple might completely remove the deprecated version on short
notice.

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

The need for a user id requires us to run 'id -u' which works on
POSIX systems but not Windows. Further, the need for fully-qualitifed
path names including $HOME behaves differently in the Git internals and
the external test suite. The $HOME variable starts with "C:\..." instead
of the "/c/..." that is provided by Git in these subcommands. The test
therefore has a prerequisite that we are not on Windows. The cross-
platform logic still allows us to test the macOS logic on a Linux
machine.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_MAINT_SCHEDULER.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format. This is useful for any system that might contain
the tool, so use it whenever it is available.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  40 +++++++
 builtin/gc.c                      | 188 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  58 +++++++++
 3 files changed, 285 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1aa1112418..5f8f63f098 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process does not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to schedule timed jobs in macOS. Scheduling
+maintenance through `git maintenance (start|stop)` requires some
+`launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/org.git-scm.git*
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+launchctl.plist(5) for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 18ae7f7138..782769f243 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *launchctl_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *launchctl_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static char *launchctl_get_uid(void)
+{
+	return xstrfmt("gui/%d", getuid());
+}
+
+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	char *uid = launchctl_get_uid();
+
+	strvec_split(&child.args, cmd);
+	if (enable)
+		strvec_push(&child.args, "bootstrap");
+	else
+		strvec_push(&child.args, "bootout");
+	strvec_push(&child.args, uid);
+	strvec_push(&child.args, filename);
+
+	child.no_stderr = 1;
+	child.no_stdout = 1;
+
+	if (start_command(&child))
+		die(_("failed to start launchctl"));
+
+	result = finish_command(&child);
+
+	free(uid);
+	return result;
+}
+
+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+	int result = launchctl_boot_plist(0, filename, cmd);
+	unlink(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int launchctl_remove_plists(const char *cmd)
+{
+	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = xfopen(filename, "w");
+
+	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+	fclose(plist);
+
+	/* bootout might fail if not already running, so ignore */
+	launchctl_boot_plist(0, filename, cmd);
+	if (launchctl_boot_plist(1, filename, cmd))
+		die(_("failed to bootstrap service %s"), filename);
+
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int launchctl_add_plists(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return launchctl_add_plists(cmd);
+	else
+		return launchctl_remove_plists(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+#if defined(__APPLE__)
+static const char platform_scheduler[] = "launchctl";
+#else
 static const char platform_scheduler[] = "crontab";
+#endif
 
 static int update_background_schedule(int enable)
 {
@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "crontab"))
+	if (!strcmp(scheduler, "launchctl"))
+		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
 		die("unknown background scheduler: %s", scheduler);
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index eeb939168d..6d37312901 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -7,6 +7,19 @@ test_description='git maintenance builtin'
 GIT_TEST_COMMIT_GRAPH=0
 GIT_TEST_MULTI_PACK_INDEX=0
 
+test_lazy_prereq XMLLINT '
+	xmllint --version
+'
+
+test_xmllint () {
+	if test_have_prereq XMLLINT
+	then
+		xmllint --noout "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success !MINGW 'start and stop macOS maintenance' '
+	uid=$(id -u) &&
+
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		test_xmllint "$PLIST" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$uid $PLIST" >>expect &&
+		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args &&
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	test_line_count = 0 actual
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

[PATCH v5 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
specify 'schtasks' as the scheduler, we can test the Windows-specific
logic on other platforms. Thus, add a check that the XML file written
by Git is valid when xmllint exists on the system.

Since we use a temporary file for the XML files sent to 'schtasks', we
must copy the file to a predictable filename. Use the number of lines in
the 'args' file to provide a filename for xmllint. Instead of an exact
match on the 'args' file, we 'grep' for the arguments other than the
filename.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 160 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  44 ++++++++
 3 files changed, 226 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 5f8f63f098..6970f2b898 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see
 launchctl.plist(5) for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 782769f243..43224e0dec 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1671,6 +1671,162 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
 		return launchctl_remove_plists(cmd);
 }
 
+static char *schtasks_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	strvec_split(&args, cmd);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int schtasks_remove_tasks(const char *cmd)
+{
+	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
+		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *xml;
+	struct tempfile *tfile;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	tfile = xmks_tempfile("schedule_XXXXXX");
+	if (!tfile || !fdopen_tempfile(tfile, "w"))
+		die(_("failed to create temp xml file"));
+
+	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fputs(xml, tfile->fp);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml = "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", tfile->filename.buf, NULL);
+	close_tempfile_gently(tfile);
+
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+
+	if (start_command(&child))
+		die(_("failed to start schtasks"));
+	result = finish_command(&child);
+
+	delete_tempfile(&tfile);
+	free(name);
+	return result;
+}
+
+static int schtasks_schedule_tasks(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return schtasks_schedule_tasks(cmd);
+	else
+		return schtasks_remove_tasks(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1761,6 +1917,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 
 #if defined(__APPLE__)
 static const char platform_scheduler[] = "launchctl";
+#elif defined(GIT_WINDOWS_NATIVE)
+static const char platform_scheduler[] = "schtasks";
 #else
 static const char platform_scheduler[] = "crontab";
 #endif
@@ -1789,6 +1947,8 @@ static int update_background_schedule(int enable)
 
 	if (!strcmp(scheduler, "launchctl"))
 		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "schtasks"))
+		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
 	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 6d37312901..0246e4ce30 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -453,6 +453,50 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '
 	test_line_count = 0 actual
 '
 
+test_expect_success 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	while test $# -gt 0
+	do
+		case "$1" in
+		/xml) shift; xmlfile=$1; break ;;
+		*) shift ;;
+		esac
+	done
+	lines=$(wc -l args | awk "{print \$1;}")
+	test -z "$xmlfile" || cp "$xmlfile" "schedule-$lines.xml"
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	for frequency in hourly daily weekly
+	do
+		grep "/create /tn Git Maintenance ($frequency) /f /xml" args \
+			|| return 1
+	done &&
+
+	for i in 1 2 3
+	do
+		test_xmllint "schedule-$i.xml" &&
+		grep "encoding=.US-ASCII." "schedule-$i.xml" || return 1
+	done &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v5 4/4] maintenance: use Windows scheduled tasks

[Eric Sunshine]

On Mon, Nov 23, 2020 at 11:16 PM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> Git's background maintenance uses cron by default, but this is not
> available on Windows. Instead, integrate with Task Scheduler.
> [...]
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> diff --git a/builtin/gc.c b/builtin/gc.c
> @@ -1671,6 +1671,162 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
> +static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
> +{
> +       const char *frequency = get_frequency(schedule);
> +
> +       tfile = xmks_tempfile("schedule_XXXXXX");
> +       if (!tfile || !fdopen_tempfile(tfile, "w"))
> +               die(_("failed to create temp xml file"));

Several comments:

The "x" prefix on xmks_tempfile() means that it will die() if it can't
open the tempfile, so the `!tfile` condition is pointless, thus it
could be written:

    if (!fdopen_tempfile(tfile, "w"))

The mks_tempfile_t*() functions with a trailing "t" will place the
temporary file in TMPDIR, whereas xmks_tempfile() used here places it
in the worktree directory, which is not as desirable. Ideally, this
would be using xmks_tempfile_t(), however, that function doesn't exist
yet in tempfile.h, so the best you can do (without the extra work of
also adding the missing function) is to use mks_tempfile_t(). That
doesn't die(), so `!tfile` would still be needed in the conditional.

In earlier versions, you incorporated `frequency` into the temporary
filename which was nice because it made the test easier to understand.
It's not hard to do so here, as well, nor to extract a useful filename
in the test (as I'll show below). For instance:

    struct strbuf tpath = STRBUF_INIT;
    strbuf_addf(&tpath, "schedule-%s-XXXXXX", frequency);
    tfile = mks_tempfile_t(tpath.buf);
    strbuf_release(&tpath);
    if (!tfile || !fdopen(tempfile(tfile, "w"))
        die(...);

> +       strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", tfile->filename.buf, NULL);

Alternately, use the getter:

    strvec_pushl(..., get_tempfile_path(&tfile), ...);

> diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
> @@ -453,6 +453,50 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '
> +test_expect_success 'start and stop Windows maintenance' '
> +       write_script print-args <<-\EOF &&
> +       echo $* >>args
> +       while test $# -gt 0
> +       do
> +               case "$1" in
> +               /xml) shift; xmlfile=$1; break ;;
> +               *) shift ;;
> +               esac
> +       done
> +       lines=$(wc -l args | awk "{print \$1;}")

You're using `awk` to pluck out the line count and ignore the
filename, but the same can be accomplished by feeding the file as
stdin to `wc` rather than naming it as an argument, thus this is
equivalent:

    lines=$(wc -l <args)

However, this idea of constructing stable names for the files by
assigning them numerically incrementing values is unnecessary and
makes the test harder to understand.

> +       test -z "$xmlfile" || cp "$xmlfile" "schedule-$lines.xml"

If you take the suggestion earlier in this review of naming the file
"schedule-${frequency}-XXXXXX.xml", then you can strip it down to just
"schedule-${frequency}.xml" using the expression `${xmlfile%-*}.xml`.
There is no need for `$lines`. Thus, copying the file would become:

    test -z "$xmlfile" || cp "$xmlfile" "${xmlfile%-*}.xml"

> +       for i in 1 2 3
> +       do

Which means that you can use the more easily understood `hourly daily
weekly` enumeration here rather than `1 2 3`.

Having said all this, I'm not sure it's worth a re-roll. These sort of
tweaks can be done atop the current series if someone wants to tackle
it.

[PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.


Update in V6
============

 * The Windows platform uses the tempfile API a bit better, including using
   the frequency in the filename to make the test simpler.

Thanks, -Stolee

cc: jrnieder@gmail.com cc: jonathantanmy@google.com cc: sluongng@gmail.com
cc: Đoàn Trần Công Danh congdanhqx@gmail.com cc: Martin Ågren
martin.agren@gmail.com cc: Eric Sunshine sunshine@sunshineco.com cc: Derrick
Stolee stolee@gmail.com

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 116 ++++++++
 builtin/gc.c                      | 421 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 105 +++++++-
 t/test-lib.sh                     |   7 +-
 4 files changed, 615 insertions(+), 34 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v5:

 1:  4807342b001 = 1:  4807342b001 maintenance: extract platform-specific scheduling
 2:  7cc70a8fe7b = 2:  7cc70a8fe7b maintenance: include 'cron' details in docs
 3:  cd015a5cbd7 = 3:  cd015a5cbd7 maintenance: use launchctl on macOS
 4:  ac9a28bea39 ! 4:  6ad4a6b98c6 maintenance: use Windows scheduled tasks
     @@ Commit message
          by Git is valid when xmllint exists on the system.
      
          Since we use a temporary file for the XML files sent to 'schtasks', we
     -    must copy the file to a predictable filename. Use the number of lines in
     -    the 'args' file to provide a filename for xmllint. Instead of an exact
     -    match on the 'args' file, we 'grep' for the arguments other than the
     -    filename.
     +    prefix the random characters with the frequency so it is easier to
     +    examine the proper file during tests. Instead of an exact match on the
     +    'args' file, we 'grep' for the arguments other than the filename.
      
          There is a deficiency in the current design. Windows has two kinds of
          applications: GUI applications that start by "winmain()" and console
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	struct tempfile *tfile;
      +	const char *frequency = get_frequency(schedule);
      +	char *name = schtasks_task_name(frequency);
     ++	struct strbuf tfilename = STRBUF_INIT;
      +
     -+	tfile = xmks_tempfile("schedule_XXXXXX");
     -+	if (!tfile || !fdopen_tempfile(tfile, "w"))
     ++	strbuf_addf(&tfilename, "schedule_%s_XXXXXX", frequency);
     ++	tfile = xmks_tempfile(tfilename.buf);
     ++	strbuf_release(&tfilename);
     ++
     ++	if (!fdopen_tempfile(tfile, "w"))
      +		die(_("failed to create temp xml file"));
      +
      +	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	      "</Task>\n";
      +	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
      +	strvec_split(&child.args, cmd);
     -+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", tfile->filename.buf, NULL);
     ++	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml",
     ++				  get_tempfile_path(tfile), NULL);
      +	close_tempfile_gently(tfile);
      +
      +	child.no_stdout = 1;
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +		*) shift ;;
      +		esac
      +	done
     -+	lines=$(wc -l args | awk "{print \$1;}")
     -+	test -z "$xmlfile" || cp "$xmlfile" "schedule-$lines.xml"
     ++	test -z "$xmlfile" || cp "$xmlfile" "$xmlfile.xml"
      +	EOF
      +
      +	rm -f args &&
     -+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" GIT_TRACE2_PERF=1 git maintenance start &&
      +
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
      +	for frequency in hourly daily weekly
      +	do
     -+		grep "/create /tn Git Maintenance ($frequency) /f /xml" args \
     -+			|| return 1
     -+	done &&
     -+
     -+	for i in 1 2 3
     -+	do
     -+		test_xmllint "schedule-$i.xml" &&
     -+		grep "encoding=.US-ASCII." "schedule-$i.xml" || return 1
     ++		grep "/create /tn Git Maintenance ($frequency) /f /xml" args &&
     ++		file=$(ls schedule_$frequency*.xml) &&
     ++		test_xmllint "$file" &&
     ++		grep "encoding=.US-ASCII." "$file" || return 1
      +	done &&
      +
      +	rm -f args &&

-- 
gitgitgadget

[PATCH v6 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

As we add this generality, rename GIT_TEST_CRONTAB to
GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as
"<scheduler>:<command>" so we can test platform-specific scheduling
logic even when not on the correct platform. By specifying the
<scheduler> in this string, we will be able to test all three sets of
Git logic from a Linux machine.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 70 ++++++++++++++++++++++++++----------------
 t/t7900-maintenance.sh |  8 ++---
 t/test-lib.sh          |  7 +++--
 3 files changed, 51 insertions(+), 34 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a12..18ae7f7138a 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
 	FILE *cron_list, *cron_in;
-	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
-
-	crontab_name = getenv("GIT_TEST_CRONTAB");
-	if (!crontab_name)
-		crontab_name = "crontab";
-
-	strvec_split(&crontab_list.args, crontab_name);
+	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
-	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_list))
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 
 	/* Ignore exit code, as an empty crontab will return error. */
 	finish_command(&crontab_list);
@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
-	strvec_split(&crontab_edit.args, crontab_name);
+	strvec_split(&crontab_edit.args, cmd);
 	crontab_edit.in = -1;
 	crontab_edit.git_cmd = 0;
 
-	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_edit))
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 
 	cron_in = fdopen(crontab_edit.in, "w");
 	if (!cron_in) {
@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static const char platform_scheduler[] = "crontab";
+
+static int update_background_schedule(int enable)
+{
+	int result;
+	const char *scheduler = platform_scheduler;
+	const char *cmd = scheduler;
+	char *testing;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	if (testing) {
+		char *sep = strchr(testing, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+		scheduler = testing;
+		cmd = sep + 1;
 	}
-	fclose(cron_list);
 
-cleanup:
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	if (!strcmp(scheduler, "crontab"))
+		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
+	else
+		die("unknown background scheduler: %s", scheduler);
+
 	rollback_lock_file(&lk);
+	free(testing);
 	return result;
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1a..eeb939168da 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '
 '
 
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '
 '
 
 test_expect_success 'stop from existing schedule' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
 	# Operation is idempotent
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 	test_must_be_empty cron.txt
 '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed766..ddbeee1f5eb 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '
 '
 
 # Ensure that no test accidentally triggers a Git command
-# that runs 'crontab', affecting a user's cron schedule.
-# Tests that verify the cron integration must set this locally
+# that runs the actual maintenance scheduler, affecting a user's
+# system permanently.
+# Tests that verify the scheduler integration must set this locally
 # to avoid errors.
-GIT_TEST_CRONTAB="exit 1"
+GIT_TEST_MAINT_SCHEDULER="none:exit 1"
-- 
gitgitgadget

[PATCH v6 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc2..1aa11124186 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is cron(8). This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `crontab` entry specifies the full path of the `git` executable to
+ensure that the executed `git` command is the same one with which
+`git maintenance start` was issued independent of `PATH`. If the same user
+runs `git maintenance start` with multiple Git executables, then only the
+latest executable is used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config. The `git maintenance` process
+then determines which maintenance tasks are configured to run on each
+repository with each `<frequency>` using the `maintenance.<task>.schedule`
+config options. These values are loaded from the global or repository
+config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+the crontab(5) documentation for advanced scheduling techniques. Please
+do use the full path and `--exec-path` techniques from the default
+schedule to ensure you are executing the correct binaries in your
+schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

[PATCH v6 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2]. The current design does not preclude a future version that
detects the available fatures of 'launchctl' to use the older
commands. However, it is best to rely on the newest version since
Apple might completely remove the deprecated version on short
notice.

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

The need for a user id requires us to run 'id -u' which works on
POSIX systems but not Windows. Further, the need for fully-qualitifed
path names including $HOME behaves differently in the Git internals and
the external test suite. The $HOME variable starts with "C:\..." instead
of the "/c/..." that is provided by Git in these subcommands. The test
therefore has a prerequisite that we are not on Windows. The cross-
platform logic still allows us to test the macOS logic on a Linux
machine.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_MAINT_SCHEDULER.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format. This is useful for any system that might contain
the tool, so use it whenever it is available.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  40 +++++++
 builtin/gc.c                      | 188 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  58 +++++++++
 3 files changed, 285 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1aa11124186..5f8f63f0988 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process does not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to schedule timed jobs in macOS. Scheduling
+maintenance through `git maintenance (start|stop)` requires some
+`launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/org.git-scm.git*
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+launchctl.plist(5) for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 18ae7f7138a..782769f2438 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *launchctl_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *launchctl_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static char *launchctl_get_uid(void)
+{
+	return xstrfmt("gui/%d", getuid());
+}
+
+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	char *uid = launchctl_get_uid();
+
+	strvec_split(&child.args, cmd);
+	if (enable)
+		strvec_push(&child.args, "bootstrap");
+	else
+		strvec_push(&child.args, "bootout");
+	strvec_push(&child.args, uid);
+	strvec_push(&child.args, filename);
+
+	child.no_stderr = 1;
+	child.no_stdout = 1;
+
+	if (start_command(&child))
+		die(_("failed to start launchctl"));
+
+	result = finish_command(&child);
+
+	free(uid);
+	return result;
+}
+
+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+	int result = launchctl_boot_plist(0, filename, cmd);
+	unlink(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int launchctl_remove_plists(const char *cmd)
+{
+	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = xfopen(filename, "w");
+
+	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+	fclose(plist);
+
+	/* bootout might fail if not already running, so ignore */
+	launchctl_boot_plist(0, filename, cmd);
+	if (launchctl_boot_plist(1, filename, cmd))
+		die(_("failed to bootstrap service %s"), filename);
+
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int launchctl_add_plists(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return launchctl_add_plists(cmd);
+	else
+		return launchctl_remove_plists(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+#if defined(__APPLE__)
+static const char platform_scheduler[] = "launchctl";
+#else
 static const char platform_scheduler[] = "crontab";
+#endif
 
 static int update_background_schedule(int enable)
 {
@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "crontab"))
+	if (!strcmp(scheduler, "launchctl"))
+		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
 		die("unknown background scheduler: %s", scheduler);
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index eeb939168da..6d373129016 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -7,6 +7,19 @@ test_description='git maintenance builtin'
 GIT_TEST_COMMIT_GRAPH=0
 GIT_TEST_MULTI_PACK_INDEX=0
 
+test_lazy_prereq XMLLINT '
+	xmllint --version
+'
+
+test_xmllint () {
+	if test_have_prereq XMLLINT
+	then
+		xmllint --noout "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -395,6 +408,51 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success !MINGW 'start and stop macOS maintenance' '
+	uid=$(id -u) &&
+
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		test_xmllint "$PLIST" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/$uid $PLIST" >>expect &&
+		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args &&
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	test_line_count = 0 actual
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

[PATCH v6 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
specify 'schtasks' as the scheduler, we can test the Windows-specific
logic on other platforms. Thus, add a check that the XML file written
by Git is valid when xmllint exists on the system.

Since we use a temporary file for the XML files sent to 'schtasks', we
prefix the random characters with the frequency so it is easier to
examine the proper file during tests. Instead of an exact match on the
'args' file, we 'grep' for the arguments other than the filename.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 165 ++++++++++++++++++++++++++++++
 t/t7900-maintenance.sh            |  39 +++++++
 3 files changed, 226 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 5f8f63f0988..6970f2b8983 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see
 launchctl.plist(5) for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 782769f2438..7c989904671 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1671,6 +1671,167 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
 		return launchctl_remove_plists(cmd);
 }
 
+static char *schtasks_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	strvec_split(&args, cmd);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int schtasks_remove_tasks(const char *cmd)
+{
+	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
+		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *xml;
+	struct tempfile *tfile;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+	struct strbuf tfilename = STRBUF_INIT;
+
+	strbuf_addf(&tfilename, "schedule_%s_XXXXXX", frequency);
+	tfile = xmks_tempfile(tfilename.buf);
+	strbuf_release(&tfilename);
+
+	if (!fdopen_tempfile(tfile, "w"))
+		die(_("failed to create temp xml file"));
+
+	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fputs(xml, tfile->fp);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml = "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml",
+				  get_tempfile_path(tfile), NULL);
+	close_tempfile_gently(tfile);
+
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+
+	if (start_command(&child))
+		die(_("failed to start schtasks"));
+	result = finish_command(&child);
+
+	delete_tempfile(&tfile);
+	free(name);
+	return result;
+}
+
+static int schtasks_schedule_tasks(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return schtasks_schedule_tasks(cmd);
+	else
+		return schtasks_remove_tasks(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1761,6 +1922,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 
 #if defined(__APPLE__)
 static const char platform_scheduler[] = "launchctl";
+#elif defined(GIT_WINDOWS_NATIVE)
+static const char platform_scheduler[] = "schtasks";
 #else
 static const char platform_scheduler[] = "crontab";
 #endif
@@ -1789,6 +1952,8 @@ static int update_background_schedule(int enable)
 
 	if (!strcmp(scheduler, "launchctl"))
 		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "schtasks"))
+		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
 	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 6d373129016..f080c29a61d 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -453,6 +453,45 @@ test_expect_success !MINGW 'start and stop macOS maintenance' '
 	test_line_count = 0 actual
 '
 
+test_expect_success 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	while test $# -gt 0
+	do
+		case "$1" in
+		/xml) shift; xmlfile=$1; break ;;
+		*) shift ;;
+		esac
+	done
+	test -z "$xmlfile" || cp "$xmlfile" "$xmlfile.xml"
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" GIT_TRACE2_PERF=1 git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	for frequency in hourly daily weekly
+	do
+		grep "/create /tn Git Maintenance ($frequency) /f /xml" args &&
+		file=$(ls schedule_$frequency*.xml) &&
+		test_xmllint "$file" &&
+		grep "encoding=.US-ASCII." "$file" || return 1
+	done &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	rm expect &&
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

"Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:

> This is based on ds/maintenance-part-3.
>
> After sitting with the background maintenance as it has been cooking, I
> wanted to come back around and implement the background maintenance for
> Windows. However, I noticed that there were some things bothering me with
> background maintenance on my macOS machine. These are detailed in PATCH 3,
> but the tl;dr is that 'cron' is not recommended by Apple and instead
> 'launchd' satisfies our needs.
>
> This series implements the background scheduling so git maintenance
> (start|stop) works on those platforms. I've been operating with these
> schedules for a while now without the problems described in the patches.
>
> There is a particularly annoying case about console windows popping up on
> Windows, but PATCH 4 describes a plan to get around that.
>
>
> Update in V6
> ============
>
>  * The Windows platform uses the tempfile API a bit better, including using
>    the frequency in the filename to make the test simpler.

Are two fix-up patches from Eric that have been queued near the top
of ds/maintenance-part-4 still relevant?  

At least, the "when invoked individually" patch that added an "-f"
option to two invocations of "rm" is still applicable, I would
think (I didn't look at the other one).

commit e3801c41e4d4cb1dd899942e04ab78310e781d07
Author: Eric Sunshine <sunshine@sunshineco.com>

    t7900: make macOS-specific test work on Windows

Notes (amlog):
    Message-Id: <20201130044224.12298-3-sunshine@sunshineco.com>

commit 1e5ddd79e2da18ee19b665a045d4187c5dc6234e
Author: Eric Sunshine <sunshine@sunshineco.com>

    t7900: fix test failures when invoked individually via --run

Notes (amlog):
    Message-Id: <20201130044224.12298-2-sunshine@sunshineco.com>

Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance

[Eric Sunshine]

On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano <gitster@pobox.com> wrote:
> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
> > Update in V6
> > ============
> >
> >  * The Windows platform uses the tempfile API a bit better, including using
> >    the frequency in the filename to make the test simpler.
>
> Are two fix-up patches from Eric that have been queued near the top
> of ds/maintenance-part-4 still relevant?

Both of the patches from Sunshine are still relevant atop Stolee's
latest (v6), and they should apply cleanly (I would think) since v6
didn't change anything related to those patches.

Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano <gitster@pobox.com> wrote:
>> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>> > Update in V6
>> > ============
>> >
>> >  * The Windows platform uses the tempfile API a bit better, including using
>> >    the frequency in the filename to make the test simpler.
>>
>> Are two fix-up patches from Eric that have been queued near the top
>> of ds/maintenance-part-4 still relevant?
>
> Both of the patches from Sunshine are still relevant atop Stolee's
> latest (v6), and they should apply cleanly (I would think) since v6
> didn't change anything related to those patches.

Yup, I tried rebasing these two and they applied cleanly, so I'll
include them in today's pushout (which I haven't finished yet).

I probably would not notice if the updated 4-patch series already
solved the issue in another way without causing the textual conflict
with your two fix-up patches, though ;-)

Thanks.

Re: [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee]

On 12/9/2020 8:04 PM, Junio C Hamano wrote:
> Eric Sunshine <sunshine@sunshineco.com> writes:
> 
>> On Wed, Dec 9, 2020 at 7:33 PM Junio C Hamano <gitster@pobox.com> wrote:
>>> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>>> Update in V6
>>>> ============
>>>>
>>>>  * The Windows platform uses the tempfile API a bit better, including using
>>>>    the frequency in the filename to make the test simpler.
>>>
>>> Are two fix-up patches from Eric that have been queued near the top
>>> of ds/maintenance-part-4 still relevant?
>>
>> Both of the patches from Sunshine are still relevant atop Stolee's
>> latest (v6), and they should apply cleanly (I would think) since v6
>> didn't change anything related to those patches.
> 
> Yup, I tried rebasing these two and they applied cleanly, so I'll
> include them in today's pushout (which I haven't finished yet).
> 
> I probably would not notice if the updated 4-patch series already
> solved the issue in another way without causing the textual conflict
> with your two fix-up patches, though ;-)

I noticed a subtle issue with the v6 series, so I _will_ reroll the
series squashing in Eric's patches. He will remain a co-author and
I'll add the Helped-by: Ævar along with the details for that patch.

Thanks,
-Stolee

[PATCH v7 0/4] Maintenance IV: Platform-specific background maintenance

[Derrick Stolee via GitGitGadget]

This is based on ds/maintenance-part-3.

After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.

This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.

There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.


Update in V7
============

 * I had included an "encoding" string in the XML file for schtasks based on
   an example using UTF-8. The cross-platform tests then complained (in
   xmllint) because they wrote in ASCII instead. However, actually testing
   the situation on Windows (see [1]) against the real schtasks finds that
   it doesn't like that encoding string. I removed it entirely, and
   everything seems happier.

 * I squashed Eric's two commits making the tests better. He remains a
   co-author and I kept his Helped-by. I had to rearrange the commit message
   a bit to point out the care he took for the cross-platform tests without
   referring to the test doing the wrong thing.

[1] https://github.com/microsoft/git/pull/304

Thanks, -Stolee

cc: jrnieder@gmail.com cc: jonathantanmy@google.com cc: sluongng@gmail.com
cc: Đoàn Trần Công Danh congdanhqx@gmail.com cc: Martin Ågren
martin.agren@gmail.com cc: Eric Sunshine sunshine@sunshineco.com cc: Derrick
Stolee stolee@gmail.com

Derrick Stolee (4):
  maintenance: extract platform-specific scheduling
  maintenance: include 'cron' details in docs
  maintenance: use launchctl on macOS
  maintenance: use Windows scheduled tasks

 Documentation/git-maintenance.txt | 116 ++++++++
 builtin/gc.c                      | 422 ++++++++++++++++++++++++++++--
 t/t7900-maintenance.sh            | 104 +++++++-
 t/test-lib.sh                     |   7 +-
 4 files changed, 615 insertions(+), 34 deletions(-)


base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/776

Range-diff vs v6:

 1:  4807342b001 = 1:  4807342b001 maintenance: extract platform-specific scheduling
 2:  7cc70a8fe7b = 2:  7cc70a8fe7b maintenance: include 'cron' details in docs
 3:  cd015a5cbd7 ! 3:  3576c7aa54e maintenance: use launchctl on macOS
     @@ Commit message
          the XML format. This is useful for any system that might contain
          the tool, so use it whenever it is available.
      
     +    We strive to make these tests work on all platforms, but Windows caused
     +    some headaches. In particular, the value of getuid() called by the C
     +    code is not guaranteed to be the same as `$(id -u)` invoked by a test.
     +    This is because `git.exe` is a native Windows program, whereas the
     +    utility programs run by the test script mostly utilize the MSYS2 runtime,
     +    which emulates a POSIX-like environment. Since the purpose of the test
     +    is to check that the input to the hook is well-formed, the actual user
     +    ID is immaterial, thus we can work around the problem by making the the
     +    test UID-agnostic. Another subtle issue is the $HOME environment
     +    variable being a Windows-style path instead of a Unix-style path. We can
     +    be more flexible here instead of expecting exact path matches.
     +
     +    Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
          Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
     @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'
       	grep "Important information!" cron.txt
       '
       
     -+test_expect_success !MINGW 'start and stop macOS maintenance' '
     -+	uid=$(id -u) &&
     ++test_expect_success 'start and stop macOS maintenance' '
     ++	# ensure $HOME can be compared against hook arguments on all platforms
     ++	pfx=$(cd "$HOME" && pwd) &&
      +
      +	write_script print-args <<-\EOF &&
     -+	echo $* >>args
     ++	echo $* | sed "s:gui/[0-9][0-9]*:gui/[UID]:" >>args
      +	EOF
      +
      +	rm -f args &&
     @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'
      +	EOF
      +	test_cmp expect actual &&
      +
     -+	rm expect &&
     ++	rm -f expect &&
      +	for frequency in hourly daily weekly
      +	do
     -+		PLIST="$HOME/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
     ++		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
      +		test_xmllint "$PLIST" &&
      +		grep schedule=$frequency "$PLIST" &&
     -+		echo "bootout gui/$uid $PLIST" >>expect &&
     -+		echo "bootstrap gui/$uid $PLIST" >>expect || return 1
     ++		echo "bootout gui/[UID] $PLIST" >>expect &&
     ++		echo "bootstrap gui/[UID] $PLIST" >>expect || return 1
      +	done &&
      +	test_cmp expect args &&
      +
     @@ t/t7900-maintenance.sh: test_expect_success 'start preserves existing schedule'
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	printf "bootout gui/$uid $HOME/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
     ++	printf "bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
      +		hourly daily weekly >expect &&
      +	test_cmp expect args &&
      +	ls "$HOME/Library/LaunchAgents" >actual &&
 4:  6ad4a6b98c6 ! 4:  68f5013dee3 maintenance: use Windows scheduled tasks
     @@ Documentation/git-maintenance.txt: To create more advanced customizations to you
       Part of the linkgit:git[1] suite
      
       ## builtin/gc.c ##
     +@@ builtin/gc.c: static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
     + 		die(_("failed to create directories for '%s'"), filename);
     + 	plist = xfopen(filename, "w");
     + 
     +-	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
     ++	preamble = "<?xml version=\"1.0\"?>\n"
     + 		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
     + 		   "<plist version=\"1.0\">"
     + 		   "<dict>\n"
      @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
       		return launchctl_remove_plists(cmd);
       }
     @@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
      +	char *name = schtasks_task_name(frequency);
      +	struct strbuf tfilename = STRBUF_INIT;
      +
     -+	strbuf_addf(&tfilename, "schedule_%s_XXXXXX", frequency);
     ++	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
     ++		    get_git_common_dir(), frequency);
      +	tfile = xmks_tempfile(tfilename.buf);
      +	strbuf_release(&tfilename);
      +
      +	if (!fdopen_tempfile(tfile, "w"))
      +		die(_("failed to create temp xml file"));
      +
     -+	xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
     ++	xml = "<?xml version=\"1.0\" ?>\n"
      +	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
      +	      "<Triggers>\n"
      +	      "<CalendarTrigger>\n";
     @@ builtin/gc.c: static int update_background_schedule(int enable)
       	else
      
       ## t/t7900-maintenance.sh ##
     -@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS maintenance' '
     +@@ t/t7900-maintenance.sh: test_expect_success 'start and stop macOS maintenance' '
       	test_line_count = 0 actual
       '
       
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +	EOF
      +
      +	rm -f args &&
     -+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" GIT_TRACE2_PERF=1 git maintenance start &&
     ++	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
      +
      +	# start registers the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +	for frequency in hourly daily weekly
      +	do
      +		grep "/create /tn Git Maintenance ($frequency) /f /xml" args &&
     -+		file=$(ls schedule_$frequency*.xml) &&
     -+		test_xmllint "$file" &&
     -+		grep "encoding=.US-ASCII." "$file" || return 1
     ++		file=$(ls .git/schedule_${frequency}*.xml) &&
     ++		test_xmllint "$file" || return 1
      +	done &&
      +
      +	rm -f args &&
     @@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
      +	# stop does not unregister the repo
      +	git config --get --global maintenance.repo "$(pwd)" &&
      +
     -+	rm expect &&
      +	printf "/delete /tn Git Maintenance (%s) /f\n" \
      +		hourly daily weekly >expect &&
      +	test_cmp expect args

-- 
gitgitgadget

[PATCH v7 1/4] maintenance: extract platform-specific scheduling

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing schedule mechanism using 'cron' is supported by POSIX
platforms, but not Windows. It also works slightly differently on
macOS to significant detriment of the user experience. To allow for
new implementations on these platforms, extract a method that
performs the platform-specific scheduling mechanism. This will be
swapped at compile time with new implementations on specialized
platforms.

As we add this generality, rename GIT_TEST_CRONTAB to
GIT_TEST_MAINT_SCHEDULER. Further, this variable is now parsed as
"<scheduler>:<command>" so we can test platform-specific scheduling
logic even when not on the correct platform. By specifying the
<scheduler> in this string, we will be able to test all three sets of
Git logic from a Linux machine.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/gc.c           | 70 ++++++++++++++++++++++++++----------------
 t/t7900-maintenance.sh |  8 ++---
 t/test-lib.sh          |  7 +++--
 3 files changed, 51 insertions(+), 34 deletions(-)

diff --git a/builtin/gc.c b/builtin/gc.c
index e3098ef6a12..18ae7f7138a 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1494,35 +1494,23 @@ static int maintenance_unregister(void)
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
-static int update_background_schedule(int run_maintenance)
+static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 {
 	int result = 0;
 	int in_old_region = 0;
 	struct child_process crontab_list = CHILD_PROCESS_INIT;
 	struct child_process crontab_edit = CHILD_PROCESS_INIT;
 	FILE *cron_list, *cron_in;
-	const char *crontab_name;
 	struct strbuf line = STRBUF_INIT;
-	struct lock_file lk;
-	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
 
-	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
-		return error(_("another process is scheduling background maintenance"));
-
-	crontab_name = getenv("GIT_TEST_CRONTAB");
-	if (!crontab_name)
-		crontab_name = "crontab";
-
-	strvec_split(&crontab_list.args, crontab_name);
+	strvec_split(&crontab_list.args, cmd);
 	strvec_push(&crontab_list.args, "-l");
 	crontab_list.in = -1;
-	crontab_list.out = dup(lk.tempfile->fd);
+	crontab_list.out = dup(fd);
 	crontab_list.git_cmd = 0;
 
-	if (start_command(&crontab_list)) {
-		result = error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_list))
+		return error(_("failed to run 'crontab -l'; your system might not support 'cron'"));
 
 	/* Ignore exit code, as an empty crontab will return error. */
 	finish_command(&crontab_list);
@@ -1531,17 +1519,15 @@ static int update_background_schedule(int run_maintenance)
 	 * Read from the .lock file, filtering out the old
 	 * schedule while appending the new schedule.
 	 */
-	cron_list = fdopen(lk.tempfile->fd, "r");
+	cron_list = fdopen(fd, "r");
 	rewind(cron_list);
 
-	strvec_split(&crontab_edit.args, crontab_name);
+	strvec_split(&crontab_edit.args, cmd);
 	crontab_edit.in = -1;
 	crontab_edit.git_cmd = 0;
 
-	if (start_command(&crontab_edit)) {
-		result = error(_("failed to run 'crontab'; your system might not support 'cron'"));
-		goto cleanup;
-	}
+	if (start_command(&crontab_edit))
+		return error(_("failed to run 'crontab'; your system might not support 'cron'"));
 
 	cron_in = fdopen(crontab_edit.in, "w");
 	if (!cron_in) {
@@ -1586,14 +1572,44 @@ static int update_background_schedule(int run_maintenance)
 	close(crontab_edit.in);
 
 done_editing:
-	if (finish_command(&crontab_edit)) {
+	if (finish_command(&crontab_edit))
 		result = error(_("'crontab' died"));
-		goto cleanup;
+	else
+		fclose(cron_list);
+	return result;
+}
+
+static const char platform_scheduler[] = "crontab";
+
+static int update_background_schedule(int enable)
+{
+	int result;
+	const char *scheduler = platform_scheduler;
+	const char *cmd = scheduler;
+	char *testing;
+	struct lock_file lk;
+	char *lock_path = xstrfmt("%s/schedule", the_repository->objects->odb->path);
+
+	testing = xstrdup_or_null(getenv("GIT_TEST_MAINT_SCHEDULER"));
+	if (testing) {
+		char *sep = strchr(testing, ':');
+		if (!sep)
+			die("GIT_TEST_MAINT_SCHEDULER unparseable: %s", testing);
+		*sep = '\0';
+		scheduler = testing;
+		cmd = sep + 1;
 	}
-	fclose(cron_list);
 
-cleanup:
+	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
+		return error(_("another process is scheduling background maintenance"));
+
+	if (!strcmp(scheduler, "crontab"))
+		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
+	else
+		die("unknown background scheduler: %s", scheduler);
+
 	rollback_lock_file(&lk);
+	free(testing);
 	return result;
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 20184e96e1a..eeb939168da 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -368,7 +368,7 @@ test_expect_success 'register and unregister' '
 '
 
 test_expect_success 'start from empty cron table' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 
 	# start registers the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
@@ -379,19 +379,19 @@ test_expect_success 'start from empty cron table' '
 '
 
 test_expect_success 'stop from existing schedule' '
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 
 	# stop does not unregister the repo
 	git config --get --global maintenance.repo "$(pwd)" &&
 
 	# Operation is idempotent
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance stop &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance stop &&
 	test_must_be_empty cron.txt
 '
 
 test_expect_success 'start preserves existing schedule' '
 	echo "Important information!" >cron.txt &&
-	GIT_TEST_CRONTAB="test-tool crontab cron.txt" git maintenance start &&
+	GIT_TEST_MAINT_SCHEDULER="crontab:test-tool crontab cron.txt" git maintenance start &&
 	grep "Important information!" cron.txt
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4a60d1ed766..ddbeee1f5eb 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1704,7 +1704,8 @@ test_lazy_prereq REBASE_P '
 '
 
 # Ensure that no test accidentally triggers a Git command
-# that runs 'crontab', affecting a user's cron schedule.
-# Tests that verify the cron integration must set this locally
+# that runs the actual maintenance scheduler, affecting a user's
+# system permanently.
+# Tests that verify the scheduler integration must set this locally
 # to avoid errors.
-GIT_TEST_CRONTAB="exit 1"
+GIT_TEST_MAINT_SCHEDULER="none:exit 1"
-- 
gitgitgadget

[PATCH v7 2/4] maintenance: include 'cron' details in docs

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Advanced and expert users may want to know how 'git maintenance start'
schedules background maintenance in order to customize their own
schedules beyond what the maintenance.* config values allow. Start a new
set of sections in git-maintenance.txt that describe how 'cron' is used
to run these tasks.

This is particularly valuable for users who want to inspect what Git is
doing or for users who want to customize the schedule further. Having a
baseline can provide a way forward for users who have never worked with
cron schedules.

Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt | 54 +++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 6fec1eb8dc2..1aa11124186 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -218,6 +218,60 @@ Further, the `git gc` command should not be combined with
 but does not take the lock in the same way as `git maintenance run`. If
 possible, use `git maintenance run --task=gc` instead of `git gc`.
 
+The following sections describe the mechanisms put in place to run
+background maintenance by `git maintenance start` and how to customize
+them.
+
+BACKGROUND MAINTENANCE ON POSIX SYSTEMS
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
+is cron(8). This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
+-----------------------------------------------------------------------
+# BEGIN GIT MAINTENANCE SCHEDULE
+# The following schedule was created by Git
+# Any edits made in this region might be
+# replaced in the future by a Git command.
+
+0 1-23 * * * "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=hourly
+0 0 * * 1-6 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=daily
+0 0 * * 0 "/<path>/git" --exec-path="/<path>" for-each-repo --config=maintenance.repo maintenance run --schedule=weekly
+
+# END GIT MAINTENANCE SCHEDULE
+-----------------------------------------------------------------------
+
+The comments are used as a region to mark the schedule as written by Git.
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
+The `crontab` entry specifies the full path of the `git` executable to
+ensure that the executed `git` command is the same one with which
+`git maintenance start` was issued independent of `PATH`. If the same user
+runs `git maintenance start` with multiple Git executables, then only the
+latest executable is used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
+loaded from the user-specific global config. The `git maintenance` process
+then determines which maintenance tasks are configured to run on each
+repository with each `<frequency>` using the `maintenance.<task>.schedule`
+config options. These values are loaded from the global or repository
+config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
+the crontab(5) documentation for advanced scheduling techniques. Please
+do use the full path and `--exec-path` techniques from the default
+schedule to ensure you are executing the correct binaries in your
+schedule.
+
 
 GIT
 ---
-- 
gitgitgadget

[PATCH v7 3/4] maintenance: use launchctl on macOS

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

The existing mechanism for scheduling background maintenance is done
through cron. The 'crontab -e' command allows updating the schedule
while cron itself runs those commands. While this is technically
supported by macOS, it has some significant deficiencies:

1. Every run of 'crontab -e' must request elevated privileges through
   the user interface. When running 'git maintenance start' from the
   Terminal app, it presents a dialog box saying "Terminal.app would
   like to administer your computer. Administration can include
   modifying passwords, networking, and system settings." This is more
   alarming than what we are hoping to achieve. If this alert had some
   information about how "git" is trying to run "crontab" then we would
   have some reason to believe that this dialog might be fine. However,
   it also doesn't help that some scenarios just leave Git waiting for
   a response without presenting anything to the user. I experienced
   this when executing the command from a Bash terminal view inside
   Visual Studio Code.

2. While cron initializes a user environment enough for "git config
   --global --show-origin" to show the correct config file information,
   it does not set up the environment enough for Git Credential Manager
   Core to load credentials during a 'prefetch' task. My prefetches
   against private repositories required re-authenticating through UI
   pop-ups in a way that should not be required.

The solution is to switch from cron to the Apple-recommended [1]
'launchd' tool.

[1] https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/ScheduledJobs.html

The basics of this tool is that we need to create XML-formatted
"plist" files inside "~/Library/LaunchAgents/" and then use the
'launchctl' tool to make launchd aware of them. The plist files
include all of the scheduling information, along with the command-line
arguments split across an array of <string> tags.

For example, here is my plist file for the weekly scheduled tasks:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict>
<key>Label</key><string>org.git-scm.git.weekly</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/git-core/git</string>
<string>--exec-path=/usr/local/libexec/git-core</string>
<string>for-each-repo</string>
<string>--config=maintenance.repo</string>
<string>maintenance</string>
<string>run</string>
<string>--schedule=weekly</string>
</array>
<key>StartCalendarInterval</key>
<array>
<dict>
<key>Day</key><integer>0</integer>
<key>Hour</key><integer>0</integer>
<key>Minute</key><integer>0</integer>
</dict>
</array>
</dict>
</plist>

The schedules for the daily and hourly tasks are more complicated
since we need to use an array for the StartCalendarInterval with
an entry for each of the six days other than the 0th day (to avoid
colliding with the weekly task), and each of the 23 hours other
than the 0th hour (to avoid colliding with the daily task).

The "Label" value is currently filled with "org.git-scm.git.X"
where X is the frequency. We need a different plist file for each
frequency.

The launchctl command needs to be aligned with a user id in order
to initialize the command environment. This must be done using
the 'launchctl bootstrap' subcommand. This subcommand is new as
of macOS 10.11, which was released in September 2015. Before that
release the 'launchctl load' subcommand was recommended. The best
source of information on this transition I have seen is available
at [2]. The current design does not preclude a future version that
detects the available fatures of 'launchctl' to use the older
commands. However, it is best to rely on the newest version since
Apple might completely remove the deprecated version on short
notice.

[2] https://babodee.wordpress.com/2016/04/09/launchctl-2-0-syntax/

To remove a schedule, we must run 'launchctl bootout' with a valid
plist file. We also need to 'bootout' a task before the 'bootstrap'
subcommand will succeed, if such a task already exists.

The need for a user id requires us to run 'id -u' which works on
POSIX systems but not Windows. Further, the need for fully-qualitifed
path names including $HOME behaves differently in the Git internals and
the external test suite. The $HOME variable starts with "C:\..." instead
of the "/c/..." that is provided by Git in these subcommands. The test
therefore has a prerequisite that we are not on Windows. The cross-
platform logic still allows us to test the macOS logic on a Linux
machine.

We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
command-line arguments into GIT_TEST_MAINT_SCHEDULER.

An earlier version of this patch accidentally had an opening
"<dict>" tag when it should have had a closing "</dict>" tag. This
was caught during manual testing with actual 'launchctl' commands,
but we do not want to update developers' tasks when running tests.
It appears that macOS includes the "xmllint" tool which can verify
the XML format. This is useful for any system that might contain
the tool, so use it whenever it is available.

We strive to make these tests work on all platforms, but Windows caused
some headaches. In particular, the value of getuid() called by the C
code is not guaranteed to be the same as `$(id -u)` invoked by a test.
This is because `git.exe` is a native Windows program, whereas the
utility programs run by the test script mostly utilize the MSYS2 runtime,
which emulates a POSIX-like environment. Since the purpose of the test
is to check that the input to the hook is well-formed, the actual user
ID is immaterial, thus we can work around the problem by making the the
test UID-agnostic. Another subtle issue is the $HOME environment
variable being a Windows-style path instead of a Unix-style path. We can
be more flexible here instead of expecting exact path matches.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  40 +++++++
 builtin/gc.c                      | 188 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  59 ++++++++++
 3 files changed, 286 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 1aa11124186..5f8f63f0988 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -273,6 +273,46 @@ schedule to ensure you are executing the correct binaries in your
 schedule.
 
 
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
+---------------------------------------
+
+While macOS technically supports `cron`, using `crontab -e` requires
+elevated privileges and the executed process does not have a full user
+context. Without a full user context, Git and its credential helpers
+cannot access stored credentials, so some maintenance tasks are not
+functional.
+
+Instead, `git maintenance start` interacts with the `launchctl` tool,
+which is the recommended way to schedule timed jobs in macOS. Scheduling
+maintenance through `git maintenance (start|stop)` requires some
+`launchctl` features available only in macOS 10.11 or later.
+
+Your user-specific scheduled tasks are stored as XML-formatted `.plist`
+files in `~/Library/LaunchAgents/`. You can see the currently-registered
+tasks using the following command:
+
+-----------------------------------------------------------------------
+$ ls ~/Library/LaunchAgents/org.git-scm.git*
+org.git-scm.git.daily.plist
+org.git-scm.git.hourly.plist
+org.git-scm.git.weekly.plist
+-----------------------------------------------------------------------
+
+One task is registered for each `--schedule=<frequency>` option. To
+inspect how the XML format describes each schedule, open one of these
+`.plist` files in an editor and inspect the `<array>` element following
+the `<key>StartCalendarInterval</key>` element.
+
+`git maintenance start` will overwrite these files and register the
+tasks again with `launchctl`, so any customizations should be done by
+creating your own `.plist` files with distinct names. Similarly, the
+`git maintenance stop` command will unregister the tasks with `launchctl`
+and delete the `.plist` files.
+
+To create more advanced customizations to your background tasks, see
+launchctl.plist(5) for more information.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 18ae7f7138a..782769f2438 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1491,6 +1491,186 @@ static int maintenance_unregister(void)
 	return run_command(&config_unset);
 }
 
+static const char *get_frequency(enum schedule_priority schedule)
+{
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		return "hourly";
+	case SCHEDULE_DAILY:
+		return "daily";
+	case SCHEDULE_WEEKLY:
+		return "weekly";
+	default:
+		BUG("invalid schedule %d", schedule);
+	}
+}
+
+static char *launchctl_service_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "org.git-scm.git.%s", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static char *launchctl_service_filename(const char *name)
+{
+	char *expanded;
+	struct strbuf filename = STRBUF_INIT;
+	strbuf_addf(&filename, "~/Library/LaunchAgents/%s.plist", name);
+
+	expanded = expand_user_path(filename.buf, 1);
+	if (!expanded)
+		die(_("failed to expand path '%s'"), filename.buf);
+
+	strbuf_release(&filename);
+	return expanded;
+}
+
+static char *launchctl_get_uid(void)
+{
+	return xstrfmt("gui/%d", getuid());
+}
+
+static int launchctl_boot_plist(int enable, const char *filename, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	char *uid = launchctl_get_uid();
+
+	strvec_split(&child.args, cmd);
+	if (enable)
+		strvec_push(&child.args, "bootstrap");
+	else
+		strvec_push(&child.args, "bootout");
+	strvec_push(&child.args, uid);
+	strvec_push(&child.args, filename);
+
+	child.no_stderr = 1;
+	child.no_stdout = 1;
+
+	if (start_command(&child))
+		die(_("failed to start launchctl"));
+
+	result = finish_command(&child);
+
+	free(uid);
+	return result;
+}
+
+static int launchctl_remove_plist(enum schedule_priority schedule, const char *cmd)
+{
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+	int result = launchctl_boot_plist(0, filename, cmd);
+	unlink(filename);
+	free(filename);
+	free(name);
+	return result;
+}
+
+static int launchctl_remove_plists(const char *cmd)
+{
+	return launchctl_remove_plist(SCHEDULE_HOURLY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_DAILY, cmd) ||
+		launchctl_remove_plist(SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_schedule_plist(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	FILE *plist;
+	int i;
+	const char *preamble, *repeat;
+	const char *frequency = get_frequency(schedule);
+	char *name = launchctl_service_name(frequency);
+	char *filename = launchctl_service_filename(name);
+
+	if (safe_create_leading_directories(filename))
+		die(_("failed to create directories for '%s'"), filename);
+	plist = xfopen(filename, "w");
+
+	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+		   "<plist version=\"1.0\">"
+		   "<dict>\n"
+		   "<key>Label</key><string>%s</string>\n"
+		   "<key>ProgramArguments</key>\n"
+		   "<array>\n"
+		   "<string>%s/git</string>\n"
+		   "<string>--exec-path=%s</string>\n"
+		   "<string>for-each-repo</string>\n"
+		   "<string>--config=maintenance.repo</string>\n"
+		   "<string>maintenance</string>\n"
+		   "<string>run</string>\n"
+		   "<string>--schedule=%s</string>\n"
+		   "</array>\n"
+		   "<key>StartCalendarInterval</key>\n"
+		   "<array>\n";
+	fprintf(plist, preamble, name, exec_path, exec_path, frequency);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		repeat = "<dict>\n"
+			 "<key>Hour</key><integer>%d</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 23; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_DAILY:
+		repeat = "<dict>\n"
+			 "<key>Day</key><integer>%d</integer>\n"
+			 "<key>Hour</key><integer>0</integer>\n"
+			 "<key>Minute</key><integer>0</integer>\n"
+			 "</dict>\n";
+		for (i = 1; i <= 6; i++)
+			fprintf(plist, repeat, i);
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(plist,
+			"<dict>\n"
+			"<key>Day</key><integer>0</integer>\n"
+			"<key>Hour</key><integer>0</integer>\n"
+			"<key>Minute</key><integer>0</integer>\n"
+			"</dict>\n");
+		break;
+
+	default:
+		/* unreachable */
+		break;
+	}
+	fprintf(plist, "</array>\n</dict>\n</plist>\n");
+	fclose(plist);
+
+	/* bootout might fail if not already running, so ignore */
+	launchctl_boot_plist(0, filename, cmd);
+	if (launchctl_boot_plist(1, filename, cmd))
+		die(_("failed to bootstrap service %s"), filename);
+
+	free(filename);
+	free(name);
+	return 0;
+}
+
+static int launchctl_add_plists(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return launchctl_schedule_plist(exec_path, SCHEDULE_HOURLY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_DAILY, cmd) ||
+		launchctl_schedule_plist(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int launchctl_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return launchctl_add_plists(cmd);
+	else
+		return launchctl_remove_plists(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1579,7 +1759,11 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 	return result;
 }
 
+#if defined(__APPLE__)
+static const char platform_scheduler[] = "launchctl";
+#else
 static const char platform_scheduler[] = "crontab";
+#endif
 
 static int update_background_schedule(int enable)
 {
@@ -1603,7 +1787,9 @@ static int update_background_schedule(int enable)
 	if (hold_lock_file_for_update(&lk, lock_path, LOCK_NO_DEREF) < 0)
 		return error(_("another process is scheduling background maintenance"));
 
-	if (!strcmp(scheduler, "crontab"))
+	if (!strcmp(scheduler, "launchctl"))
+		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
 		die("unknown background scheduler: %s", scheduler);
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index eeb939168da..adf24dee72d 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -7,6 +7,19 @@ test_description='git maintenance builtin'
 GIT_TEST_COMMIT_GRAPH=0
 GIT_TEST_MULTI_PACK_INDEX=0
 
+test_lazy_prereq XMLLINT '
+	xmllint --version
+'
+
+test_xmllint () {
+	if test_have_prereq XMLLINT
+	then
+		xmllint --noout "$@"
+	else
+		true
+	fi
+}
+
 test_expect_success 'help text' '
 	test_expect_code 129 git maintenance -h 2>err &&
 	test_i18ngrep "usage: git maintenance <subcommand>" err &&
@@ -395,6 +408,52 @@ test_expect_success 'start preserves existing schedule' '
 	grep "Important information!" cron.txt
 '
 
+test_expect_success 'start and stop macOS maintenance' '
+	# ensure $HOME can be compared against hook arguments on all platforms
+	pfx=$(cd "$HOME" && pwd) &&
+
+	write_script print-args <<-\EOF &&
+	echo $* | sed "s:gui/[0-9][0-9]*:gui/[UID]:" >>args
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	cat >expect <<-\EOF &&
+	org.git-scm.git.daily.plist
+	org.git-scm.git.hourly.plist
+	org.git-scm.git.weekly.plist
+	EOF
+	test_cmp expect actual &&
+
+	rm -f expect &&
+	for frequency in hourly daily weekly
+	do
+		PLIST="$pfx/Library/LaunchAgents/org.git-scm.git.$frequency.plist" &&
+		test_xmllint "$PLIST" &&
+		grep schedule=$frequency "$PLIST" &&
+		echo "bootout gui/[UID] $PLIST" >>expect &&
+		echo "bootstrap gui/[UID] $PLIST" >>expect || return 1
+	done &&
+	test_cmp expect args &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER=launchctl:./print-args git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "bootout gui/[UID] $pfx/Library/LaunchAgents/org.git-scm.git.%s.plist\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args &&
+	ls "$HOME/Library/LaunchAgents" >actual &&
+	test_line_count = 0 actual
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget

Re: [PATCH v7 3/4] maintenance: use launchctl on macOS

[Eric Sunshine]

On Tue, Jan 5, 2021 at 8:08 AM Derrick Stolee via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> [...]
> The need for a user id requires us to run 'id -u' which works on
> POSIX systems but not Windows. Further, the need for fully-qualitifed
> path names including $HOME behaves differently in the Git internals and
> the external test suite. The $HOME variable starts with "C:\..." instead
> of the "/c/..." that is provided by Git in these subcommands. The test
> therefore has a prerequisite that we are not on Windows. The cross-
> platform logic still allows us to test the macOS logic on a Linux
> machine.

You forgot to remove the above paragraph...

> We strive to make these tests work on all platforms, but Windows caused
> some headaches. In particular, the value of getuid() called by the C
> code is not guaranteed to be the same as `$(id -u)` invoked by a test.
> This is because `git.exe` is a native Windows program, whereas the
> utility programs run by the test script mostly utilize the MSYS2 runtime,
> which emulates a POSIX-like environment. Since the purpose of the test
> is to check that the input to the hook is well-formed, the actual user
> ID is immaterial, thus we can work around the problem by making the the
> test UID-agnostic. Another subtle issue is the $HOME environment
> variable being a Windows-style path instead of a Unix-style path. We can
> be more flexible here instead of expecting exact path matches.

...when you added this paragraph from my separate patch which you
folded into this patch for v7. The two paragraphs are at odds with one
another.

[PATCH v7 4/4] maintenance: use Windows scheduled tasks

[Derrick Stolee via GitGitGadget]

From: Derrick Stolee <dstolee@microsoft.com>

Git's background maintenance uses cron by default, but this is not
available on Windows. Instead, integrate with Task Scheduler.

Tasks can be scheduled using the 'schtasks' command. There are several
command-line options that can allow for some advanced scheduling, but
unfortunately these seem to all require authenticating using a password.

Instead, use the "/xml" option to pass an XML file that contains the
configuration for the necessary schedule. These XML files are based on
some that I exported after constructing a schedule in the Task Scheduler
GUI. These options only run background maintenance when the user is
logged in, and more fields are populated with the current username and
SID at run-time by 'schtasks'.

Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
specify 'schtasks' as the scheduler, we can test the Windows-specific
logic on other platforms. Thus, add a check that the XML file written
by Git is valid when xmllint exists on the system.

Since we use a temporary file for the XML files sent to 'schtasks', we
prefix the random characters with the frequency so it is easier to
examine the proper file during tests. Instead of an exact match on the
'args' file, we 'grep' for the arguments other than the filename.

There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
to a new Console window if they are not already associated with a GUI
application. This means that every hour the scheudled task launches a
command window for the scheduled tasks. Not only is this visually
obtrusive, but it also takes focus from whatever else the user is
doing!

A simple fix would be to insert a GUI application that acts as a shim
between the scheduled task and Git. This is currently possible in Git
for Windows by setting the <Command> tag equal to

  C:\Program Files\Git\git-bash.exe

with options "--hide --no-needs-console --command=cmd\git.exe"
followed by the arguments currently used. Since git-bash.exe is not
included in Windows builds of core Git, I chose to leave out this
feature. My plan is to submit a small patch to Git for Windows that
converts the use of git.exe with this use of git-bash.exe in the
short term. In the long term, we can consider creating this GUI
shim application within core Git, perhaps in contrib/.

Co-authored-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  22 ++++
 builtin/gc.c                      | 168 +++++++++++++++++++++++++++++-
 t/t7900-maintenance.sh            |  37 +++++++
 3 files changed, 226 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 5f8f63f0988..6970f2b8983 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -313,6 +313,28 @@ To create more advanced customizations to your background tasks, see
 launchctl.plist(5) for more information.
 
 
+BACKGROUND MAINTENANCE ON WINDOWS SYSTEMS
+-----------------------------------------
+
+Windows does not support `cron` and instead has its own system for
+scheduling background tasks. The `git maintenance start` command uses
+the `schtasks` command to submit tasks to this system. You can inspect
+all background tasks using the Task Scheduler application. The tasks
+added by Git have names of the form `Git Maintenance (<frequency>)`.
+The Task Scheduler GUI has ways to inspect these tasks, but you can also
+export the tasks to XML files and view the details there.
+
+Note that since Git is a console application, these background tasks
+create a console window visible to the current user. This can be changed
+manually by selecting the "Run whether user is logged in or not" option
+in Task Scheduler. This change requires a password input, which is why
+`git maintenance start` does not select it by default.
+
+If you want to customize the background tasks, please rename the tasks
+so future calls to `git maintenance (start|stop)` do not overwrite your
+custom tasks.
+
+
 GIT
 ---
 Part of the linkgit:git[1] suite
diff --git a/builtin/gc.c b/builtin/gc.c
index 782769f2438..fdc95d9e99f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1589,7 +1589,7 @@ static int launchctl_schedule_plist(const char *exec_path, enum schedule_priorit
 		die(_("failed to create directories for '%s'"), filename);
 	plist = xfopen(filename, "w");
 
-	preamble = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+	preamble = "<?xml version=\"1.0\"?>\n"
 		   "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
 		   "<plist version=\"1.0\">"
 		   "<dict>\n"
@@ -1671,6 +1671,168 @@ static int launchctl_update_schedule(int run_maintenance, int fd, const char *cm
 		return launchctl_remove_plists(cmd);
 }
 
+static char *schtasks_task_name(const char *frequency)
+{
+	struct strbuf label = STRBUF_INIT;
+	strbuf_addf(&label, "Git Maintenance (%s)", frequency);
+	return strbuf_detach(&label, NULL);
+}
+
+static int schtasks_remove_task(enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct strvec args = STRVEC_INIT;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+
+	strvec_split(&args, cmd);
+	strvec_pushl(&args, "/delete", "/tn", name, "/f", NULL);
+
+	result = run_command_v_opt(args.v, 0);
+
+	strvec_clear(&args);
+	free(name);
+	return result;
+}
+
+static int schtasks_remove_tasks(const char *cmd)
+{
+	return schtasks_remove_task(SCHEDULE_HOURLY, cmd) ||
+		schtasks_remove_task(SCHEDULE_DAILY, cmd) ||
+		schtasks_remove_task(SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_schedule_task(const char *exec_path, enum schedule_priority schedule, const char *cmd)
+{
+	int result;
+	struct child_process child = CHILD_PROCESS_INIT;
+	const char *xml;
+	struct tempfile *tfile;
+	const char *frequency = get_frequency(schedule);
+	char *name = schtasks_task_name(frequency);
+	struct strbuf tfilename = STRBUF_INIT;
+
+	strbuf_addf(&tfilename, "%s/schedule_%s_XXXXXX",
+		    get_git_common_dir(), frequency);
+	tfile = xmks_tempfile(tfilename.buf);
+	strbuf_release(&tfilename);
+
+	if (!fdopen_tempfile(tfile, "w"))
+		die(_("failed to create temp xml file"));
+
+	xml = "<?xml version=\"1.0\" ?>\n"
+	      "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
+	      "<Triggers>\n"
+	      "<CalendarTrigger>\n";
+	fputs(xml, tfile->fp);
+
+	switch (schedule) {
+	case SCHEDULE_HOURLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T01:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByDay>\n"
+			"<DaysInterval>1</DaysInterval>\n"
+			"</ScheduleByDay>\n"
+			"<Repetition>\n"
+			"<Interval>PT1H</Interval>\n"
+			"<Duration>PT23H</Duration>\n"
+			"<StopAtDurationEnd>false</StopAtDurationEnd>\n"
+			"</Repetition>\n");
+		break;
+
+	case SCHEDULE_DAILY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Monday />\n"
+			"<Tuesday />\n"
+			"<Wednesday />\n"
+			"<Thursday />\n"
+			"<Friday />\n"
+			"<Saturday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	case SCHEDULE_WEEKLY:
+		fprintf(tfile->fp,
+			"<StartBoundary>2020-01-01T00:00:00</StartBoundary>\n"
+			"<Enabled>true</Enabled>\n"
+			"<ScheduleByWeek>\n"
+			"<DaysOfWeek>\n"
+			"<Sunday />\n"
+			"</DaysOfWeek>\n"
+			"<WeeksInterval>1</WeeksInterval>\n"
+			"</ScheduleByWeek>\n");
+		break;
+
+	default:
+		break;
+	}
+
+	xml = "</CalendarTrigger>\n"
+	      "</Triggers>\n"
+	      "<Principals>\n"
+	      "<Principal id=\"Author\">\n"
+	      "<LogonType>InteractiveToken</LogonType>\n"
+	      "<RunLevel>LeastPrivilege</RunLevel>\n"
+	      "</Principal>\n"
+	      "</Principals>\n"
+	      "<Settings>\n"
+	      "<MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n"
+	      "<Enabled>true</Enabled>\n"
+	      "<Hidden>true</Hidden>\n"
+	      "<UseUnifiedSchedulingEngine>true</UseUnifiedSchedulingEngine>\n"
+	      "<WakeToRun>false</WakeToRun>\n"
+	      "<ExecutionTimeLimit>PT72H</ExecutionTimeLimit>\n"
+	      "<Priority>7</Priority>\n"
+	      "</Settings>\n"
+	      "<Actions Context=\"Author\">\n"
+	      "<Exec>\n"
+	      "<Command>\"%s\\git.exe\"</Command>\n"
+	      "<Arguments>--exec-path=\"%s\" for-each-repo --config=maintenance.repo maintenance run --schedule=%s</Arguments>\n"
+	      "</Exec>\n"
+	      "</Actions>\n"
+	      "</Task>\n";
+	fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
+	strvec_split(&child.args, cmd);
+	strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml",
+				  get_tempfile_path(tfile), NULL);
+	close_tempfile_gently(tfile);
+
+	child.no_stdout = 1;
+	child.no_stderr = 1;
+
+	if (start_command(&child))
+		die(_("failed to start schtasks"));
+	result = finish_command(&child);
+
+	delete_tempfile(&tfile);
+	free(name);
+	return result;
+}
+
+static int schtasks_schedule_tasks(const char *cmd)
+{
+	const char *exec_path = git_exec_path();
+
+	return schtasks_schedule_task(exec_path, SCHEDULE_HOURLY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_DAILY, cmd) ||
+		schtasks_schedule_task(exec_path, SCHEDULE_WEEKLY, cmd);
+}
+
+static int schtasks_update_schedule(int run_maintenance, int fd, const char *cmd)
+{
+	if (run_maintenance)
+		return schtasks_schedule_tasks(cmd);
+	else
+		return schtasks_remove_tasks(cmd);
+}
+
 #define BEGIN_LINE "# BEGIN GIT MAINTENANCE SCHEDULE"
 #define END_LINE "# END GIT MAINTENANCE SCHEDULE"
 
@@ -1761,6 +1923,8 @@ static int crontab_update_schedule(int run_maintenance, int fd, const char *cmd)
 
 #if defined(__APPLE__)
 static const char platform_scheduler[] = "launchctl";
+#elif defined(GIT_WINDOWS_NATIVE)
+static const char platform_scheduler[] = "schtasks";
 #else
 static const char platform_scheduler[] = "crontab";
 #endif
@@ -1789,6 +1953,8 @@ static int update_background_schedule(int enable)
 
 	if (!strcmp(scheduler, "launchctl"))
 		result = launchctl_update_schedule(enable, lk.tempfile->fd, cmd);
+	else if (!strcmp(scheduler, "schtasks"))
+		result = schtasks_update_schedule(enable, lk.tempfile->fd, cmd);
 	else if (!strcmp(scheduler, "crontab"))
 		result = crontab_update_schedule(enable, lk.tempfile->fd, cmd);
 	else
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index adf24dee72d..135505f6195 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -454,6 +454,43 @@ test_expect_success 'start and stop macOS maintenance' '
 	test_line_count = 0 actual
 '
 
+test_expect_success 'start and stop Windows maintenance' '
+	write_script print-args <<-\EOF &&
+	echo $* >>args
+	while test $# -gt 0
+	do
+		case "$1" in
+		/xml) shift; xmlfile=$1; break ;;
+		*) shift ;;
+		esac
+	done
+	test -z "$xmlfile" || cp "$xmlfile" "$xmlfile.xml"
+	EOF
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance start &&
+
+	# start registers the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	for frequency in hourly daily weekly
+	do
+		grep "/create /tn Git Maintenance ($frequency) /f /xml" args &&
+		file=$(ls .git/schedule_${frequency}*.xml) &&
+		test_xmllint "$file" || return 1
+	done &&
+
+	rm -f args &&
+	GIT_TEST_MAINT_SCHEDULER="schtasks:./print-args" git maintenance stop &&
+
+	# stop does not unregister the repo
+	git config --get --global maintenance.repo "$(pwd)" &&
+
+	printf "/delete /tn Git Maintenance (%s) /f\n" \
+		hourly daily weekly >expect &&
+	test_cmp expect args
+'
+
 test_expect_success 'register preserves existing strategy' '
 	git config maintenance.strategy none &&
 	git maintenance register &&
-- 
gitgitgadget