git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Emily Shaffer <emilyshaffer@google.com>
To: git@vger.kernel.org
Cc: Emily Shaffer <emilyshaffer@google.com>
Subject: [PATCH 10/17] hook: support passing stdin to hooks
Date: Fri,  4 Dec 2020 17:46:00 -0800	[thread overview]
Message-ID: <20201205014607.1464119-11-emilyshaffer@google.com> (raw)
In-Reply-To: <20201205014607.1464119-1-emilyshaffer@google.com>

Some hooks (such as post-rewrite) need to take input via stdin.
Previously, callers provided stdin to hooks by setting
run-command.h:child_process.in, which takes a FD. Callers would open the
file in question themselves before calling run-command(). However, since
we will now need to seek to the front of the file and read it again for
every hook which runs, hook.h:run_command() takes a path and handles FD
management itself. Since this file is opened for read only, it should
not prevent later parallel execution support.

On the frontend, this is supported by asking for a file path, rather
than by reading stdin. Reading directly from stdin would involve caching
the entire stdin (to memory or to disk) and reading it back from the
beginning to each hook. We'd want to support cases like insufficient
memory or storage for the file. While this may prove useful later, for
now the path of least resistance is to just ask the user to make this
interim file themselves.

Signed-off-by: Emily Shaffer <emilyshaffer@google.com>
---
 Documentation/git-hook.txt    | 11 +++++++++--
 builtin/hook.c                |  5 ++++-
 hook.c                        |  7 ++++++-
 hook.h                        |  9 +++++++--
 t/t1360-config-based-hooks.sh | 24 ++++++++++++++++++++++++
 5 files changed, 50 insertions(+), 6 deletions(-)

diff --git a/Documentation/git-hook.txt b/Documentation/git-hook.txt
index 18a817d832..cce30a80d0 100644
--- a/Documentation/git-hook.txt
+++ b/Documentation/git-hook.txt
@@ -9,7 +9,8 @@ SYNOPSIS
 --------
 [verse]
 'git hook' list <hook-name>
-'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hook-name>
+'git hook' run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>]
+	<hook-name>
 
 DESCRIPTION
 -----------
@@ -65,7 +66,7 @@ in the order they should be run, and print the config scope where the relevant
 `hook.<hook-name>.command` was specified, not the `hookcmd` (if applicable).
 This output is human-readable and the format is subject to change over time.
 
-run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] `<hook-name>`::
+run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] [--to-stdin=<path>] `<hook-name>`::
 
 Runs hooks configured for `<hook-name>`, in the same order displayed by `git
 hook list`. Hooks configured this way are run prepended with `sh -c`, so paths
@@ -91,6 +92,12 @@ Specify arguments to pass to every hook that is run.
 +
 Specify environment variables to set for every hook that is run.
 
+--to-stdin::
+	Only valid for `run`.
++
+Specify a file which will be streamed into stdin for every hook that is run.
+Each hook will receive the entire file from beginning to EOF.
+
 CONFIGURATION
 -------------
 include::config/hook.txt[]
diff --git a/builtin/hook.c b/builtin/hook.c
index 26f7050387..e45831e01d 100644
--- a/builtin/hook.c
+++ b/builtin/hook.c
@@ -9,7 +9,8 @@
 
 static const char * const builtin_hook_usage[] = {
 	N_("git hook list <hookname>"),
-	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...] <hookname>"),
+	N_("git hook run [(-e|--env)=<var>...] [(-a|--arg)=<arg>...]"
+	   "[--to-stdin=<path>] <hookname>"),
 	NULL
 };
 
@@ -97,6 +98,8 @@ static int run(int argc, const char **argv, const char *prefix)
 			   N_("environment variables for hook to use")),
 		OPT_STRVEC('a', "arg", &opt.args, N_("args"),
 			   N_("argument to pass to hook")),
+		OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"),
+			   N_("file to read into hooks' stdin")),
 		OPT_END(),
 	};
 
diff --git a/hook.c b/hook.c
index a7a4abdcac..c7fdf556fe 100644
--- a/hook.c
+++ b/hook.c
@@ -263,8 +263,13 @@ int run_hooks(const char *hookname, struct run_hooks_opt *options)
 		struct child_process hook_proc = CHILD_PROCESS_INIT;
 		struct hook *hook = list_entry(pos, struct hook, list);
 
+		/* reopen the file for stdin; run_command closes it. */
+		if (options->path_to_stdin)
+			hook_proc.in = xopen(options->path_to_stdin, O_RDONLY);
+		else
+			hook_proc.no_stdin = 1;
+
 		hook_proc.env = options->env.v;
-		hook_proc.no_stdin = 1;
 		hook_proc.stdout_to_stderr = 1;
 		hook_proc.trace2_hook_name = hook->command.buf;
 		hook_proc.use_shell = 1;
diff --git a/hook.h b/hook.h
index 94a25c7cd0..5184dcaa5a 100644
--- a/hook.h
+++ b/hook.h
@@ -51,11 +51,15 @@ struct run_hooks_opt
 	 * to be overridden if the user can override it at the command line.
 	 */
 	enum hookdir_opt run_hookdir;
+
+	/* Path to file which should be piped to stdin for each hook */
+	const char *path_to_stdin;
 };
 
 #define RUN_HOOKS_OPT_INIT  {   		\
-	.env = STRVEC_INIT, 				\
+	.env = STRVEC_INIT, 			\
 	.args = STRVEC_INIT, 			\
+	.path_to_stdin = NULL,			\
 	.run_hookdir = configured_hookdir_opt()	\
 }
 
@@ -73,7 +77,8 @@ int hook_exists(const char *hookname, enum hookdir_opt should_run_hookdir);
 
 /*
  * Runs all hooks associated to the 'hookname' event in order. Each hook will be
- * passed 'env' and 'args'.
+ * passed 'env' and 'args'. The file at 'stdin_path' will be closed and reopened
+ * for each hook that runs.
  */
 int run_hooks(const char *hookname, struct run_hooks_opt *options);
 
diff --git a/t/t1360-config-based-hooks.sh b/t/t1360-config-based-hooks.sh
index 5b3003d59b..c672269ee4 100755
--- a/t/t1360-config-based-hooks.sh
+++ b/t/t1360-config-based-hooks.sh
@@ -229,4 +229,28 @@ test_expect_success 'out-of-repo runs excluded' '
 	nongit test_must_fail git hook run pre-commit
 '
 
+test_expect_success 'stdin to multiple hooks' '
+	git config --add hook.test.command "xargs -P1 -I% echo a%" &&
+	git config --add hook.test.command "xargs -P1 -I% echo b%" &&
+	test_when_finished "test_unconfig hook.test.command" &&
+
+	cat >input <<-EOF &&
+	1
+	2
+	3
+	EOF
+
+	cat >expected <<-EOF &&
+	a1
+	a2
+	a3
+	b1
+	b2
+	b3
+	EOF
+
+	git hook run --to-stdin=input test 2>actual &&
+	test_cmp expected actual
+'
+
 test_done
-- 
2.28.0.rc0.142.g3c755180ce-goog


  parent reply	other threads:[~2020-12-05  1:57 UTC|newest]

Thread overview: 170+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-21 18:54 [PATCH v2 0/4] propose config-based hooks Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 1/4] doc: propose hooks managed by the config Emily Shaffer
2020-05-22 10:13   ` Phillip Wood
2020-06-09 20:26     ` Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 2/4] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 3/4] hook: add list command Emily Shaffer
2020-05-22 10:27   ` Phillip Wood
2020-06-09 21:49     ` Emily Shaffer
2020-08-17 13:36       ` Phillip Wood
2020-05-24 23:00   ` Johannes Schindelin
2020-05-27 23:37     ` Emily Shaffer
2020-05-21 18:54 ` [PATCH v2 4/4] hook: add --porcelain to " Emily Shaffer
2020-05-24 23:00   ` Johannes Schindelin
2020-05-25  0:29     ` Johannes Schindelin
2020-07-28 22:24 ` [PATCH v3 0/6] propose config-based hooks Emily Shaffer
2020-07-28 22:24   ` [PATCH v3 1/6] doc: propose hooks managed by the config Emily Shaffer
2020-07-28 22:24   ` [PATCH v3 2/6] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-07-28 22:24   ` [PATCH v3 3/6] hook: add list command Emily Shaffer
2020-07-28 22:24   ` [PATCH v3 4/6] hook: add --porcelain to " Emily Shaffer
2020-07-28 22:24   ` [RFC PATCH v3 5/6] parse-options: parse into argv_array Emily Shaffer
2020-07-29 19:33     ` Junio C Hamano
2020-07-30 23:41       ` Junio C Hamano
2020-07-28 22:24   ` [RFC PATCH v3 6/6] hook: add 'run' subcommand Emily Shaffer
2020-09-09  0:49   ` [PATCH v4 0/9] propose config-based hooks Emily Shaffer
2020-09-09  0:49     ` [PATCH v4 1/9] doc: propose hooks managed by the config Emily Shaffer
2020-09-23 22:59       ` Jonathan Tan
2020-09-24 21:54         ` Emily Shaffer
2020-10-07  9:23       ` Ævar Arnfjörð Bjarmason
2020-10-22  0:58         ` Emily Shaffer
2020-10-23 19:10           ` Ævar Arnfjörð Bjarmason
2020-10-29 15:38             ` Emily Shaffer
2020-10-29 20:04               ` Ævar Arnfjörð Bjarmason
2020-09-09  0:49     ` [PATCH v4 2/9] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-10-05 23:24       ` Jonathan Nieder
2020-10-06 19:06         ` Emily Shaffer
2020-09-09  0:49     ` [PATCH v4 3/9] hook: add list command Emily Shaffer
2020-09-11 13:27       ` Phillip Wood
2020-09-11 16:51         ` Emily Shaffer
2020-09-23 23:04       ` Jonathan Tan
2020-10-06 20:46         ` Emily Shaffer
2020-09-27 19:23       ` Martin Ågren
2020-10-06 20:20         ` Emily Shaffer
2020-10-05 23:27       ` Jonathan Nieder
2020-09-09  0:49     ` [PATCH v4 4/9] hook: add --porcelain to " Emily Shaffer
2020-09-28 19:29       ` Josh Steadmon
2020-09-09  0:49     ` [PATCH v4 5/9] parse-options: parse into strvec Emily Shaffer
2020-10-05 23:30       ` Jonathan Nieder
2020-10-06  4:49         ` Junio C Hamano
2020-09-09  0:49     ` [PATCH v4 6/9] hook: add 'run' subcommand Emily Shaffer
2020-09-11 13:30       ` Phillip Wood
2020-09-28 19:29       ` Josh Steadmon
2020-10-05 23:39       ` Jonathan Nieder
2020-10-06 22:57         ` Emily Shaffer
2020-09-09  0:49     ` [PATCH v4 7/9] hook: replace run-command.h:find_hook Emily Shaffer
2020-09-09 20:32       ` Junio C Hamano
2020-09-10 19:08         ` Emily Shaffer
2020-09-23 23:20       ` Jonathan Tan
2020-10-05 23:42       ` Jonathan Nieder
2020-09-09  0:49     ` [PATCH v4 8/9] commit: use config-based hooks Emily Shaffer
2020-09-10 13:50       ` Phillip Wood
2020-09-10 22:21         ` Junio C Hamano
2020-09-23 23:47       ` Jonathan Tan
2020-10-05 21:27         ` Emily Shaffer
2020-10-05 23:48           ` Jonathan Nieder
2020-10-06 19:08             ` Emily Shaffer
2020-09-09  0:49     ` [PATCH v4 9/9] run_commit_hook: take strvec instead of varargs Emily Shaffer
2020-09-10 14:16       ` Phillip Wood
2020-09-11 13:20         ` Phillip Wood
2020-09-09 21:04     ` [PATCH v4 0/9] propose config-based hooks Junio C Hamano
2020-10-14 23:24     ` [PATCH v5 0/8] propose config-based hooks (part I) Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 1/8] doc: propose hooks managed by the config Emily Shaffer
2020-10-15 16:31         ` Ævar Arnfjörð Bjarmason
2020-10-16 17:29           ` Junio C Hamano
2020-10-21 23:37           ` Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 2/8] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 3/8] hook: add list command Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 4/8] hook: include hookdir hook in list Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 5/8] hook: implement hookcmd.<name>.skip Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 6/8] parse-options: parse into strvec Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 7/8] hook: add 'run' subcommand Emily Shaffer
2020-10-14 23:24       ` [PATCH v5 8/8] hook: replace find_hook() with hook_exists() Emily Shaffer
2020-12-05  1:45       ` [PATCH v6 00/17] propose config-based hooks (part I) Emily Shaffer
2020-12-05  1:45         ` [PATCH 01/17] doc: propose hooks managed by the config Emily Shaffer
2020-12-05  1:45         ` [PATCH 02/17] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-12-05  1:45         ` [PATCH 03/17] hook: add list command Emily Shaffer
2020-12-05  1:45         ` [PATCH 04/17] hook: include hookdir hook in list Emily Shaffer
2020-12-05  1:45         ` [PATCH 05/17] hook: respect hook.runHookDir Emily Shaffer
2020-12-05  1:45         ` [PATCH 06/17] hook: implement hookcmd.<name>.skip Emily Shaffer
2020-12-05  1:45         ` [PATCH 07/17] parse-options: parse into strvec Emily Shaffer
2020-12-05  1:45         ` [PATCH 08/17] hook: add 'run' subcommand Emily Shaffer
2020-12-11 10:15           ` Phillip Wood
2020-12-15 21:41             ` Emily Shaffer
2020-12-05  1:45         ` [PATCH 09/17] hook: replace find_hook() with hook_exists() Emily Shaffer
2020-12-05  1:46         ` Emily Shaffer [this message]
2020-12-05  1:46         ` [PATCH 11/17] run-command: allow stdin for run_processes_parallel Emily Shaffer
2020-12-05  1:46         ` [PATCH 12/17] hook: allow parallel hook execution Emily Shaffer
2020-12-05  1:46         ` [PATCH 13/17] hook: allow specifying working directory for hooks Emily Shaffer
2020-12-05  1:46         ` [PATCH 14/17] run-command: add stdin callback for parallelization Emily Shaffer
2020-12-05  1:46         ` [PATCH 15/17] hook: provide stdin by string_list or callback Emily Shaffer
2020-12-08 21:09           ` SZEDER Gábor
2020-12-08 22:11             ` Emily Shaffer
2020-12-05  1:46         ` [PATCH 16/17] run-command: allow capturing of collated output Emily Shaffer
2020-12-05  1:46         ` [PATCH 17/17] hooks: allow callers to capture output Emily Shaffer
2020-12-16  0:34         ` [PATCH v6 00/17] propose config-based hooks (part I) Josh Steadmon
2020-12-16  0:56           ` Junio C Hamano
2020-12-16 20:16             ` Emily Shaffer
2020-12-16 23:32               ` Junio C Hamano
2020-12-18  2:07                 ` Emily Shaffer
2020-12-18  5:29                   ` Junio C Hamano
2020-12-22  0:02         ` [PATCH v7 " Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 01/17] doc: propose hooks managed by the config Emily Shaffer
2021-01-23 15:38             ` Ævar Arnfjörð Bjarmason
2021-01-29 23:52               ` Emily Shaffer
2021-02-01 22:11             ` Junio C Hamano
2021-03-10 19:30               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 02/17] hook: scaffolding for git-hook subcommand Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 03/17] hook: add list command Emily Shaffer
2021-01-31  3:10             ` Jonathan Tan
2021-02-09 21:06               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 04/17] hook: include hookdir hook in list Emily Shaffer
2021-01-31  3:20             ` Jonathan Tan
2021-02-09 22:05               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 05/17] hook: respect hook.runHookDir Emily Shaffer
2021-01-31  3:35             ` Jonathan Tan
2021-02-09 22:31               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 06/17] hook: implement hookcmd.<name>.skip Emily Shaffer
2021-01-31  3:40             ` Jonathan Tan
2021-02-09 22:57               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 07/17] parse-options: parse into strvec Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 08/17] hook: add 'run' subcommand Emily Shaffer
2021-01-31  4:22             ` Jonathan Tan
2021-02-11 22:44               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 09/17] hook: replace find_hook() with hook_exists() Emily Shaffer
2021-01-31  4:39             ` Jonathan Tan
2021-02-12 22:15               ` Emily Shaffer
2021-02-18 22:23               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 10/17] hook: support passing stdin to hooks Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 11/17] run-command: allow stdin for run_processes_parallel Emily Shaffer
2021-02-01  5:38             ` Jonathan Tan
2021-02-19 20:23               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 12/17] hook: allow parallel hook execution Emily Shaffer
2021-02-01  6:04             ` Jonathan Tan
2021-02-22 21:46               ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 13/17] hook: allow specifying working directory for hooks Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 14/17] run-command: add stdin callback for parallelization Emily Shaffer
2021-02-01  6:51             ` Jonathan Tan
2021-02-22 23:38               ` Emily Shaffer
2021-02-23 19:33                 ` Jonathan Tan
2021-03-10 18:24                   ` Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 15/17] hook: provide stdin by string_list or callback Emily Shaffer
2021-02-01  7:04             ` Jonathan Tan
2021-02-23 19:52               ` Emily Shaffer
2021-02-25 20:56                 ` Jonathan Tan
2021-03-02  1:47                   ` Emily Shaffer
2021-03-02 23:33                     ` Jonathan Tan
2020-12-22  0:02           ` [PATCH v7 16/17] run-command: allow capturing of collated output Emily Shaffer
2020-12-22  0:02           ` [PATCH v7 17/17] hooks: allow callers to capture output Emily Shaffer
2020-12-22  2:11           ` [PATCH v7 00/17] propose config-based hooks (part I) Junio C Hamano
2020-12-28 18:34             ` Emily Shaffer
2020-12-28 22:50               ` Junio C Hamano
2020-12-28 22:37           ` [PATCH v3 18/17] doc: make git-hook.txt point of truth Emily Shaffer
2020-12-28 22:39             ` Emily Shaffer
2021-01-29 23:59           ` [PATCH v7 00/17] propose config-based hooks (part I) Emily Shaffer
2021-02-16 19:46           ` Josh Steadmon
2021-02-16 22:47             ` Junio C Hamano
2021-02-17 21:21               ` Josh Steadmon
2021-02-17 23:07                 ` Junio C Hamano
2021-02-25 19:50                   ` Junio C Hamano
2021-03-01 21:51                     ` Emily Shaffer
2021-03-01 22:19                       ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201205014607.1464119-11-emilyshaffer@google.com \
    --to=emilyshaffer@google.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).