From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: jrnieder@gmail.com, jonathantanmy@google.com, sluongng@gmail.com,
"Đoàn Trần Công Danh" <congdanhqx@gmail.com>,
"Martin Ågren" <martin.agren@gmail.com>,
"Eric Sunshine" <sunshine@sunshineco.com>,
"Derrick Stolee" <stolee@gmail.com>,
"Derrick Stolee" <derrickstolee@github.com>
Subject: [PATCH v5 0/4] Maintenance IV: Platform-specific background maintenance
Date: Tue, 24 Nov 2020 04:16:41 +0000 [thread overview]
Message-ID: <pull.776.v5.git.1606191405.gitgitgadget@gmail.com> (raw)
In-Reply-To: <pull.776.v4.git.1605647598.gitgitgadget@gmail.com>
This is based on ds/maintenance-part-3.
After sitting with the background maintenance as it has been cooking, I
wanted to come back around and implement the background maintenance for
Windows. However, I noticed that there were some things bothering me with
background maintenance on my macOS machine. These are detailed in PATCH 3,
but the tl;dr is that 'cron' is not recommended by Apple and instead
'launchd' satisfies our needs.
This series implements the background scheduling so git maintenance
(start|stop) works on those platforms. I've been operating with these
schedules for a while now without the problems described in the patches.
There is a particularly annoying case about console windows popping up on
Windows, but PATCH 4 describes a plan to get around that.
Updates in V5
=============
* Fixed docs from PATCH 2 to match those in v3.
* Despite my best efforts, I was unable to make the macOS tests work on
Windows, so they are still marked with "!MINGW". I updated the commit
message to describe my problems there.
* The Windows platform now uses xmks_tempfile() to create the XML files for
'schtasks'. This led to some test fallout since the pathnames are no
longer predictable.
Thanks, -Stolee
Derrick Stolee (4):
maintenance: extract platform-specific scheduling
maintenance: include 'cron' details in docs
maintenance: use launchctl on macOS
maintenance: use Windows scheduled tasks
Documentation/git-maintenance.txt | 116 +++++++++
builtin/gc.c | 416 ++++++++++++++++++++++++++++--
t/t7900-maintenance.sh | 110 +++++++-
t/test-lib.sh | 7 +-
4 files changed, 615 insertions(+), 34 deletions(-)
base-commit: 0016b618182f642771dc589cf0090289f9fe1b4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-776%2Fderrickstolee%2Fmaintenance%2FmacOS-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-776/derrickstolee/maintenance/macOS-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/776
Range-diff vs v4:
1: 4807342b00 = 1: 4807342b00 maintenance: extract platform-specific scheduling
2: 99170df462 ! 2: 7cc70a8fe7 maintenance: include 'cron' details in docs
@@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
+---------------------------------------
+
+The standard mechanism for scheduling background tasks on POSIX systems
-+is `cron`. This tool executes commands based on a given schedule. The
++is cron(8). This tool executes commands based on a given schedule. The
+current list of user-scheduled tasks can be found by running `crontab -l`.
+The schedule written by `git maintenance start` is similar to this:
+
@@ Documentation/git-maintenance.txt: Further, the `git gc` command should not be c
+Any modifications within this region will be completely deleted by
+`git maintenance stop` or overwritten by `git maintenance start`.
+
-+The `<path>` string is loaded to specifically use the location for the
-+`git` executable used in the `git maintenance start` command. This allows
-+for multiple versions to be compatible. However, if the same user runs
-+`git maintenance start` with multiple Git executables, then only the
-+latest executable will be used.
++The `crontab` entry specifies the full path of the `git` executable to
++ensure that the executed `git` command is the same one with which
++`git maintenance start` was issued independent of `PATH`. If the same user
++runs `git maintenance start` with multiple Git executables, then only the
++latest executable is used.
+
+These commands use `git for-each-repo --config=maintenance.repo` to run
+`git maintenance run --schedule=<frequency>` on each repository listed in
+the multi-valued `maintenance.repo` config option. These are typically
-+loaded from the user-specific global config located at `~/.gitconfig`.
-+The `git maintenance` process then determines which maintenance tasks
-+are configured to run on each repository with each `<frequency>` using
-+the `maintenance.<task>.schedule` config options. These values are loaded
-+from the global or repository config values.
++loaded from the user-specific global config. The `git maintenance` process
++then determines which maintenance tasks are configured to run on each
++repository with each `<frequency>` using the `maintenance.<task>.schedule`
++config options. These values are loaded from the global or repository
++config values.
+
+If the config values are insufficient to achieve your desired background
+maintenance schedule, then you can create your own schedule. If you run
+`crontab -e`, then an editor will load with your user-specific `cron`
+schedule. In that editor, you can add your own schedule lines. You could
+start by adapting the default schedule listed earlier, or you could read
-+https://man7.org/linux/man-pages/man5/crontab.5.html[the `crontab` documentation]
-+for advanced scheduling techniques. Please do use the full path and
-+`--exec-path` techniques from the default schedule to ensure you are
-+executing the correct binaries in your schedule.
++the crontab(5) documentation for advanced scheduling techniques. Please
++do use the full path and `--exec-path` techniques from the default
++schedule to ensure you are executing the correct binaries in your
++schedule.
+
GIT
3: ed0a0011fb ! 3: cd015a5cbd maintenance: use launchctl on macOS
@@ Commit message
subcommand will succeed, if such a task already exists.
The need for a user id requires us to run 'id -u' which works on
- POSIX systems but not Windows. The test therefore has a prerequisite
- that we are not on Windows. The cross-platform logic still allows us to
- test the macOS logic on a Linux machine.
+ POSIX systems but not Windows. Further, the need for fully-qualitifed
+ path names including $HOME behaves differently in the Git internals and
+ the external test suite. The $HOME variable starts with "C:\..." instead
+ of the "/c/..." that is provided by Git in these subcommands. The test
+ therefore has a prerequisite that we are not on Windows. The cross-
+ platform logic still allows us to test the macOS logic on a Linux
+ machine.
We can verify the commands that were run by 'git maintenance start'
and 'git maintenance stop' by injecting a script that writes the
@@ Commit message
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
## Documentation/git-maintenance.txt ##
-@@ Documentation/git-maintenance.txt: for advanced scheduling techniques. Please do use the full path and
- executing the correct binaries in your schedule.
+@@ Documentation/git-maintenance.txt: schedule to ensure you are executing the correct binaries in your
+ schedule.
+BACKGROUND MAINTENANCE ON MACOS SYSTEMS
4: b8d86fb983 ! 4: ac9a28bea3 maintenance: use Windows scheduled tasks
@@ Commit message
Since the GIT_TEST_MAINT_SCHEDULER environment variable allows us to
specify 'schtasks' as the scheduler, we can test the Windows-specific
- logic on a macOS platform. Thus, add a check that the XML file written
+ logic on other platforms. Thus, add a check that the XML file written
by Git is valid when xmllint exists on the system.
+ Since we use a temporary file for the XML files sent to 'schtasks', we
+ must copy the file to a predictable filename. Use the number of lines in
+ the 'args' file to provide a filename for xmllint. Instead of an exact
+ match on the 'args' file, we 'grep' for the arguments other than the
+ filename.
+
There is a deficiency in the current design. Windows has two kinds of
applications: GUI applications that start by "winmain()" and console
applications that start by "main()". Console applications are attached
@@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
+ int result;
+ struct child_process child = CHILD_PROCESS_INIT;
+ const char *xml;
-+ char *xmlpath;
+ struct tempfile *tfile;
+ const char *frequency = get_frequency(schedule);
+ char *name = schtasks_task_name(frequency);
+
-+ xmlpath = xstrfmt("%s/schedule-%s.xml",
-+ the_repository->objects->odb->path,
-+ frequency);
-+ tfile = create_tempfile(xmlpath);
++ tfile = xmks_tempfile("schedule_XXXXXX");
+ if (!tfile || !fdopen_tempfile(tfile, "w"))
-+ die(_("failed to create '%s'"), xmlpath);
++ die(_("failed to create temp xml file"));
+
+ xml = "<?xml version=\"1.0\" encoding=\"US-ASCII\"?>\n"
+ "<Task version=\"1.4\" xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n"
@@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
+ "</Task>\n";
+ fprintf(tfile->fp, xml, exec_path, exec_path, frequency);
+ strvec_split(&child.args, cmd);
-+ strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", xmlpath, NULL);
++ strvec_pushl(&child.args, "/create", "/tn", name, "/f", "/xml", tfile->filename.buf, NULL);
+ close_tempfile_gently(tfile);
+
+ child.no_stdout = 1;
@@ builtin/gc.c: static int launchctl_update_schedule(int run_maintenance, int fd,
+ result = finish_command(&child);
+
+ delete_tempfile(&tfile);
-+ free(xmlpath);
+ free(name);
+ return result;
+}
@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
+ *) shift ;;
+ esac
+ done
-+ test -z "$xmlfile" || cp "$xmlfile" .
++ lines=$(wc -l args | awk "{print \$1;}")
++ test -z "$xmlfile" || cp "$xmlfile" "schedule-$lines.xml"
+ EOF
+
+ rm -f args &&
@@ t/t7900-maintenance.sh: test_expect_success !MINGW 'start and stop macOS mainten
+ # start registers the repo
+ git config --get --global maintenance.repo "$(pwd)" &&
+
-+ printf "/create /tn Git Maintenance (%s) /f /xml .git/objects/schedule-%s.xml\n" \
-+ hourly hourly daily daily weekly weekly >expect &&
-+ test_cmp expect args &&
-+
+ for frequency in hourly daily weekly
+ do
-+ test_xmllint "schedule-$frequency.xml"
++ grep "/create /tn Git Maintenance ($frequency) /f /xml" args \
++ || return 1
++ done &&
++
++ for i in 1 2 3
++ do
++ test_xmllint "schedule-$i.xml" &&
++ grep "encoding=.US-ASCII." "schedule-$i.xml" || return 1
+ done &&
+
+ rm -f args &&
--
gitgitgadget
next prev parent reply other threads:[~2020-11-24 4:19 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-03 14:03 [PATCH 0/3] Maintenance IV: Platform-specific background maintenance Derrick Stolee via GitGitGadget
2020-11-03 14:03 ` [PATCH 1/3] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-11-03 14:03 ` [PATCH 2/3] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-11-03 18:45 ` Eric Sunshine
2020-11-03 21:21 ` Derrick Stolee
2020-11-03 22:27 ` Eric Sunshine
2020-11-04 13:33 ` Derrick Stolee
2020-11-04 14:17 ` Derrick Stolee
2020-11-03 14:03 ` [PATCH 3/3] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-11-03 19:06 ` Eric Sunshine
2020-11-03 21:23 ` Derrick Stolee
2020-11-03 20:18 ` [PATCH 0/3] Maintenance IV: Platform-specific background maintenance Junio C Hamano
2020-11-03 20:21 ` Junio C Hamano
2020-11-03 21:09 ` Derrick Stolee
2020-11-03 22:30 ` Junio C Hamano
2020-11-04 13:02 ` Derrick Stolee
2020-11-04 17:00 ` Junio C Hamano
2020-11-04 18:43 ` Derrick Stolee
2020-11-04 20:06 ` [PATCH v2 0/4] " Derrick Stolee via GitGitGadget
2020-11-04 20:06 ` [PATCH v2 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-11-04 20:06 ` [PATCH v2 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2020-11-11 7:10 ` Eric Sunshine
2020-11-04 20:06 ` [PATCH v2 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-11-11 8:12 ` Eric Sunshine
2020-11-12 13:42 ` Derrick Stolee
2020-11-12 16:43 ` Eric Sunshine
2020-11-04 20:06 ` [PATCH v2 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-11-11 8:59 ` Eric Sunshine
2020-11-12 13:56 ` Derrick Stolee
2020-11-13 14:00 ` [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance Derrick Stolee via GitGitGadget
2020-11-13 14:00 ` [PATCH v3 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-11-13 14:00 ` [PATCH v3 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2020-11-13 14:00 ` [PATCH v3 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-11-13 20:19 ` Eric Sunshine
2020-11-13 20:42 ` Derrick Stolee
2020-11-13 20:53 ` Eric Sunshine
2020-11-13 20:56 ` Eric Sunshine
2020-11-13 14:00 ` [PATCH v3 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-11-13 20:44 ` Eric Sunshine
2020-11-13 21:32 ` Derrick Stolee
2020-11-13 21:40 ` Eric Sunshine
2020-11-16 13:13 ` Derrick Stolee
2020-11-13 20:47 ` [PATCH v3 0/4] Maintenance IV: Platform-specific background maintenance Eric Sunshine
2020-11-14 9:23 ` Eric Sunshine
2020-11-16 13:17 ` Derrick Stolee
2020-11-17 21:13 ` [PATCH v4 " Derrick Stolee via GitGitGadget
2020-11-17 21:13 ` [PATCH v4 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-11-17 21:13 ` [PATCH v4 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2020-11-18 0:34 ` Eric Sunshine
2020-11-18 18:30 ` Derrick Stolee
2020-11-17 21:13 ` [PATCH v4 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-11-18 6:45 ` Eric Sunshine
2020-11-18 18:22 ` Derrick Stolee
2020-11-17 21:13 ` [PATCH v4 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-11-18 7:15 ` Eric Sunshine
2020-11-18 18:30 ` Derrick Stolee
2020-11-18 20:54 ` Eric Sunshine
2020-11-18 21:16 ` Derrick Stolee
2020-11-17 23:36 ` [PATCH v4 0/4] Maintenance IV: Platform-specific background maintenance Eric Sunshine
2020-11-24 2:20 ` Derrick Stolee
2020-11-24 2:59 ` Eric Sunshine
2020-11-17 23:54 ` Eric Sunshine
2020-11-24 4:16 ` Derrick Stolee via GitGitGadget [this message]
2020-11-24 4:16 ` [PATCH v5 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-11-24 4:16 ` [PATCH v5 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2020-11-24 4:16 ` [PATCH v5 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-11-24 4:16 ` [PATCH v5 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-11-27 9:08 ` Eric Sunshine
2020-12-09 19:28 ` [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance Derrick Stolee via GitGitGadget
2020-12-09 19:28 ` [PATCH v6 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2020-12-09 19:29 ` [PATCH v6 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2020-12-09 19:29 ` [PATCH v6 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2020-12-09 19:29 ` [PATCH v6 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
2020-12-10 0:32 ` [PATCH v6 0/4] Maintenance IV: Platform-specific background maintenance Junio C Hamano
2020-12-10 0:49 ` Eric Sunshine
2020-12-10 1:04 ` Junio C Hamano
2021-01-05 12:17 ` Derrick Stolee
2021-01-05 13:08 ` [PATCH v7 " Derrick Stolee via GitGitGadget
2021-01-05 13:08 ` [PATCH v7 1/4] maintenance: extract platform-specific scheduling Derrick Stolee via GitGitGadget
2021-01-05 13:08 ` [PATCH v7 2/4] maintenance: include 'cron' details in docs Derrick Stolee via GitGitGadget
2021-01-05 13:08 ` [PATCH v7 3/4] maintenance: use launchctl on macOS Derrick Stolee via GitGitGadget
2021-01-10 6:34 ` Eric Sunshine
2021-01-05 13:08 ` [PATCH v7 4/4] maintenance: use Windows scheduled tasks Derrick Stolee via GitGitGadget
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pull.776.v5.git.1606191405.gitgitgadget@gmail.com \
--to=gitgitgadget@gmail.com \
--cc=congdanhqx@gmail.com \
--cc=derrickstolee@github.com \
--cc=git@vger.kernel.org \
--cc=jonathantanmy@google.com \
--cc=jrnieder@gmail.com \
--cc=martin.agren@gmail.com \
--cc=sluongng@gmail.com \
--cc=stolee@gmail.com \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).