[PATCH 0/3] refs-advertise: add hook to filter advertised refs

[PATCH 0/3] refs-advertise: add hook to filter advertised refs

[Sun Chao via GitGitGadget]

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and is possible if we add a reference advertise
filter hook just like what Gerrit did.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase and the commit fetching phase, each
reference and will be filtered by this hook. Git server can put
reference-level control process to this hook and the git client does not
need to change or known about that.

And there is also a partial clone issue says 'Dynamic object fetching'
causes the server send a full set of info/refs when the connection is
established, and if there are large number of refs, will incur significant
overhead, and if we put such kind of hook maybe it will help.

In Huawei company, we are trying to copy the reference-level control feature
from Gerrit to CGit (which is used by our git server), and we think maybe
this hook is a good idea and wish for your replies, thanks!

Sun Chao (3):
  refs-advertise: add hook to filter advertised refs
  t1419: add test cases for refs-advertise hook
  doc: add documentation for the refs-advertise hook

 Documentation/githooks.txt                    |  70 +++++
 Makefile                                      |   2 +
 builtin/receive-pack.c                        |  12 +
 ls-refs.c                                     |  30 ++-
 refs/refs-advertise.c                         | 246 +++++++++++++++++
 refs/refs-advertise.h                         |  11 +
 t/helper/test-refs-advertise.c                | 202 ++++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-refs-advertise-hooks.sh               | 138 ++++++++++
 t/t1419/common-functions.sh                   |  80 ++++++
 .../once-0000-abnormal-refs-advertise-hook.sh | 199 ++++++++++++++
 t/t1419/test-0000-standard-git-clone.sh       |  56 ++++
 t/t1419/test-0001-standard-git-push.sh        |  34 +++
 ...0002-ls-remote-with-refs-advertise-hook.sh |  95 +++++++
 ...03-upload-pack-with-refs-advertise-hook.sh | 127 +++++++++
 ...4-receive-pack-with-refs-advertise-hook.sh | 251 ++++++++++++++++++
 upload-pack.c                                 |  42 ++-
 18 files changed, 1588 insertions(+), 9 deletions(-)
 create mode 100644 refs/refs-advertise.c
 create mode 100644 refs/refs-advertise.h
 create mode 100644 t/helper/test-refs-advertise.c
 create mode 100755 t/t1419-refs-advertise-hooks.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0000-standard-git-clone.sh
 create mode 100644 t/t1419/test-0001-standard-git-push.sh
 create mode 100644 t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh


base-commit: 350dc9f0e8974b6fcbdeb3808186c5a79c3e7386
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v1
Pull-Request: https://github.com/git/git/pull/1301
-- 
gitgitgadget

[PATCH 1/3] refs-advertise: add hook to filter advertised refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and is possible if we add a reference advertise
filter hook just like what Gerrit did.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase and the commit fetching phase,
each reference and will be filtered by this hook. The hook executes once
with no arguments for each 'git-upload-pack' and 'git-receive-pack' process
and if the exit status is non-zero, `git push` and `git fetch` will abort.

Once the hook is invoked, a version number and server process name
('git-upload-pack' or 'git-receive-pack' or 'ls-refs') should send to it in
packet-line format, followed by a flush-pkt. The hook should response with
its version number and process name list it support. If the list does not
contains the server process name, the server will close the connection with
the hook and keep going without the hook child process.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be advertised or not, it sends result back in
pkt-line format protocol, a response in "ok ref <ref>" format followed by a
flush-pkt means the references "<ref>" can be advertised, and "ng ref <ref>"
means not.

    # Version negotiation
    G: PKT-LINE(version=1\0git-upload-pack)
    G: flush-pkt
    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
    H: flush-pkt

    # Send reference filter request to hook
    G: PKT-LINE(ref <ref> <oid>)
    G: flush-pkt

    # Receive result from the hook.
    # Case 1: this reference is valid
    H: PKT-LINE(ok ref <ref>)
    H: flush-pkt
    # Case 2: this reference is filtered out
    H: PKT-LINE(ng ref <ref>)
    H: flush-pkt

During commit fetch phase of 'git-upload-pack' process, git client may send
`want <oid>` requests and 'git-upload-pack' will send object filter requests
to the hook to check if the object "<oid>" will be sent to the client or
not. In the following example, the letter 'G' stands for 'git-upload-pack'
and the letter 'H' stands for this hook.

The hook will decides if a commit will be sent to the client or not, it
sends result in pkt-line format protocol to `git-upload-pack`, a response
with "ok obj <oid>" format followed by a flush-pkt means the object "<oid>"
can be sent to client, and "ng obj <oid>" means not.

    # Version negotiation
    G: PKT-LINE(version=1\0ls-refs)
    G: flush-pkt
    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
    H: flush-pkt

    # Send commit filter request to hook
    G: PKT-LINE(obj <oid>)
    G: flush-pkt

    # Receive result from the hook.
    # Case 1: this object is valid
    H: PKT-LINE(ok obj <oid>)
    H: flush-pkt
    # Case 2: this object is filtered out
    H: PKT-LINE(ng obj <oid>)
    H: flush-pkt

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                       |   2 +
 builtin/receive-pack.c         |  12 ++
 ls-refs.c                      |  30 +++-
 refs/refs-advertise.c          | 246 +++++++++++++++++++++++++++++++++
 refs/refs-advertise.h          |  11 ++
 t/helper/test-refs-advertise.c | 202 +++++++++++++++++++++++++++
 t/helper/test-tool.c           |   1 +
 t/helper/test-tool.h           |   1 +
 upload-pack.c                  |  42 +++++-
 9 files changed, 538 insertions(+), 9 deletions(-)
 create mode 100644 refs/refs-advertise.c
 create mode 100644 refs/refs-advertise.h
 create mode 100644 t/helper/test-refs-advertise.c

diff --git a/Makefile b/Makefile
index 1624471badc..38740a52cee 100644
--- a/Makefile
+++ b/Makefile
@@ -793,6 +793,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-refs-advertise.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
@@ -1050,6 +1051,7 @@ LIB_OBJS += refs/files-backend.o
 LIB_OBJS += refs/iterator.o
 LIB_OBJS += refs/packed-backend.o
 LIB_OBJS += refs/ref-cache.o
+LIB_OBJS += refs/refs-advertise.o
 LIB_OBJS += refspec.o
 LIB_OBJS += remote.o
 LIB_OBJS += replace-object.o
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31b48e728be..2f82fd05f25 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -30,6 +30,7 @@
 #include "commit-reach.h"
 #include "worktree.h"
 #include "shallow.h"
+#include "refs/refs-advertise.h"
 
 static const char * const receive_pack_usage[] = {
 	N_("git receive-pack <git-dir>"),
@@ -299,6 +300,10 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,
 	if (ref_is_hidden(path, path_full))
 		return 0;
 
+	if (filter_advertise_ref(path, oid)) {
+		return 0;
+	}
+
 	/*
 	 * Advertise refs outside our current namespace as ".have"
 	 * refs, so that the client can use them to minimize data
@@ -2458,6 +2463,10 @@ static int delete_only(struct command *commands)
 	return 1;
 }
 
+static void clean_refs_advertise_filter(void) {
+	clean_advertise_refs_filter();
+}
+
 int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 {
 	int advertise_refs = 0;
@@ -2492,6 +2501,9 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 	if (!enter_repo(service_dir, 0))
 		die("'%s' does not appear to be a git repository", service_dir);
 
+	create_advertise_refs_filter("git-receive-pack");
+	atexit(clean_refs_advertise_filter);
+
 	git_config(receive_pack_config, NULL);
 	if (cert_nonce_seed)
 		push_cert_nonce = prepare_push_cert_nonce(service_dir, time(NULL));
diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..5a1f74abe0a 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -6,10 +6,13 @@
 #include "ls-refs.h"
 #include "pkt-line.h"
 #include "config.h"
+#include "run-command.h"
+#include "refs/refs-advertise.h"
 
 static int config_read;
 static int advertise_unborn;
 static int allow_unborn;
+static struct string_list symref = STRING_LIST_INIT_DUP;
 
 static void ensure_config_read(void)
 {
@@ -81,6 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 {
 	struct ls_refs_data *data = cb_data;
 	const char *refname_nons = strip_namespace(refname);
+	const char *refname_to_filter = refname_nons;
 
 	strbuf_reset(&data->buf);
 
@@ -90,6 +94,16 @@ static int send_ref(const char *refname, const struct object_id *oid,
 	if (!ref_match(&data->prefixes, refname_nons))
 		return 0;
 
+	if (!strcmp(refname_nons, "HEAD")) {
+		struct string_list_item *item = string_list_lookup(&symref, "HEAD");
+		if (item) {
+			refname_to_filter = (const char *)item->util;
+		}
+	}
+
+	if (filter_advertise_ref(refname_to_filter, oid))
+		return 0;
+
 	if (oid)
 		strbuf_addf(&data->buf, "%s %s", oid_to_hex(oid), refname_nons);
 	else
@@ -121,18 +135,24 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 static void send_possibly_unborn_head(struct ls_refs_data *data)
 {
+	const char *symref_target;
+	struct string_list_item *item;
 	struct strbuf namespaced = STRBUF_INIT;
 	struct object_id oid;
 	int flag;
 	int oid_is_null;
 
 	strbuf_addf(&namespaced, "%sHEAD", get_git_namespace());
-	if (!resolve_ref_unsafe(namespaced.buf, 0, &oid, &flag))
+	symref_target = resolve_ref_unsafe(namespaced.buf, 0, &oid, &flag);
+	if (!symref_target)
 		return; /* bad ref */
 	oid_is_null = is_null_oid(&oid);
 	if (!oid_is_null ||
-	    (data->unborn && data->symrefs && (flag & REF_ISSYMREF)))
+	    (data->unborn && data->symrefs && (flag & REF_ISSYMREF))) {
+		item = string_list_append(&symref, "HEAD");
+		item->util = xstrdup(strip_namespace(symref_target));
 		send_ref(namespaced.buf, oid_is_null ? NULL : &oid, flag, data);
+	}
 	strbuf_release(&namespaced);
 }
 
@@ -146,6 +166,10 @@ static int ls_refs_config(const char *var, const char *value, void *data)
 	return parse_hide_refs_config(var, value, "uploadpack");
 }
 
+static void clean_refs_advertise_filter(void) {
+	clean_advertise_refs_filter();
+}
+
 int ls_refs(struct repository *r, struct packet_reader *request)
 {
 	struct ls_refs_data data;
@@ -156,6 +180,8 @@ int ls_refs(struct repository *r, struct packet_reader *request)
 
 	ensure_config_read();
 	git_config(ls_refs_config, NULL);
+	create_advertise_refs_filter("ls-refs");
+	atexit(clean_refs_advertise_filter);
 
 	while (packet_reader_read(request) == PACKET_READ_NORMAL) {
 		const char *arg = request->line;
diff --git a/refs/refs-advertise.c b/refs/refs-advertise.c
new file mode 100644
index 00000000000..0dcced78716
--- /dev/null
+++ b/refs/refs-advertise.c
@@ -0,0 +1,246 @@
+#include "../cache.h"
+#include "../config.h"
+#include "../strbuf.h"
+#include "../hook.h"
+#include "../sigchain.h"
+#include "../pkt-line.h"
+#include "../refs.h"
+#include "../run-command.h"
+#include "connect.h"
+#include "ref-cache.h"
+#include "refs-advertise.h"
+
+struct advertise_refs_filter {
+	struct child_process proc;
+	struct packet_reader reader;
+};
+
+static struct advertise_refs_filter *hook_filter = NULL;
+
+void create_advertise_refs_filter(const char *command) {
+	struct advertise_refs_filter *filter;
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int command_support = 0;
+	int version = 0;
+	int code;
+
+	if (hook_filter != NULL)
+		return;
+
+	hook_path = find_hook("refs-advertise");
+	if (!hook_path) {
+		return;
+	}
+
+	filter = (struct advertise_refs_filter *) xcalloc (1, sizeof (struct advertise_refs_filter));
+	proc = &filter->proc;
+	reader = &filter->reader;
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "refs-advertise";
+	proc->err = 0;
+
+	code = start_command(proc);
+	if (code)
+		die("can not run hook refs-advertise");
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE |
+			   PACKET_READ_GENTLE_ON_EOF);
+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', command);
+	if (!code)
+		code = packet_flush_gently(proc->in);
+
+	if (!code)
+		for (;;) {
+			int linelen;
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether refs-advertise exited abnormally */
+				if (status == PACKET_READ_EOF)
+					die("can not read version message from hook refs-advertise");
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+				linelen = strlen(reader->line);
+				if (linelen < reader->pktlen) {
+					const char *command_list = reader->line + linelen + 1;
+					if (parse_feature_request(command_list, command)) {
+						command_support = 1;
+					}
+				}
+			}
+		}
+
+	if (code)
+		die("can not read version message from hook refs-advertise");
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		die(_("hook refs-advertise version '%d' is not supported"), version);
+	}
+
+	sigchain_pop(SIGPIPE);
+
+	if (!command_support) {
+		close(proc->in);
+		close(proc->out);
+		kill(proc->pid, SIGTERM);
+		finish_command_in_signal(proc);
+		free(filter);
+
+		return;
+	}
+
+	hook_filter = filter;
+	return;
+}
+
+void clean_advertise_refs_filter(void) {
+	struct child_process *proc;
+
+	if (!hook_filter) {
+		return;
+	}
+
+	proc = &hook_filter->proc;
+
+	close(proc->in);
+	close(proc->out);
+	kill(proc->pid, SIGTERM);
+	finish_command_in_signal(proc);
+	FREE_AND_NULL(hook_filter);
+}
+
+static int do_filter_advertise_ref(const char *refname, const struct object_id *oid) {
+	struct child_process *proc;
+	struct packet_reader *reader;
+	struct strbuf buf = STRBUF_INIT;
+	char *oid_hex;
+	int code;
+
+	proc = &hook_filter->proc;
+	reader = &hook_filter->reader;
+	if (oid)
+		oid_hex = oid_to_hex(oid);
+	else
+		oid_hex = oid_to_hex(null_oid());
+
+	code = packet_write_fmt_gently(proc->in, "ref %s %s", refname, oid_hex);
+	if (code)
+		die("hook refs-advertise died abnormally");
+
+	code = packet_flush_gently(proc->in);
+	if (code)
+		die("hook refs-advertise died abnormally");
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether refs-advertise exited abnormally */
+			if (status == PACKET_READ_EOF)
+				die("hook refs-advertise died abnormally");
+			break;
+		}
+
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	if (strncmp("ok ref ", buf.buf, 7))
+		return -1;
+
+	if (strcmp(refname, buf.buf + 7))
+		return -1;
+
+	return 0;
+}
+
+int filter_advertise_ref(const char *refname, const struct object_id *oid) {
+	int result;
+
+	if (!hook_filter) {
+		return 0;
+	}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	result = do_filter_advertise_ref(refname, oid);
+	sigchain_pop(SIGPIPE);
+
+	return result;
+}
+
+static int do_filter_advertise_object(const struct object_id *oid) {
+	struct child_process *proc;
+	struct packet_reader *reader;
+	struct strbuf buf = STRBUF_INIT;
+	char *oid_hex;
+	int code;
+
+	proc = &hook_filter->proc;
+	reader = &hook_filter->reader;
+	oid_hex = oid_to_hex(oid);
+
+	code = packet_write_fmt_gently(proc->in, "obj %s", oid_hex);
+	if (code)
+		die("hook refs-advertise died abnormally");
+
+	code = packet_flush_gently(proc->in);
+	if (code)
+		die("hook refs-advertise died abnormally");
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether refs-advertise exited abnormally */
+			if (status == PACKET_READ_EOF)
+				die("hook refs-advertise died abnormally");
+			break;
+		}
+
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	if (strncmp("ok obj ", buf.buf, 7))
+		return -1;
+
+	if (strcmp(oid_hex, buf.buf + 7))
+		return -1;
+
+	return 0;
+}
+
+int filter_advertise_object(const struct object_id *oid) {
+	int result;
+
+	if (!hook_filter || !oid) {
+		return 0;
+	}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	result = do_filter_advertise_object(oid);
+	sigchain_pop(SIGPIPE);
+
+	return result;
+}
diff --git a/refs/refs-advertise.h b/refs/refs-advertise.h
new file mode 100644
index 00000000000..615469eaddb
--- /dev/null
+++ b/refs/refs-advertise.h
@@ -0,0 +1,11 @@
+#ifndef REFS_REFS_ADVERTISE_H
+#define REFS_REFS_ADVERTISE_H
+
+#include "../hash.h"
+
+void create_advertise_refs_filter(const char *command);
+void clean_advertise_refs_filter(void);
+int filter_advertise_ref(const char *refname, const struct object_id *oid);
+int filter_advertise_object(const struct object_id *oid);
+
+#endif
diff --git a/t/helper/test-refs-advertise.c b/t/helper/test-refs-advertise.c
new file mode 100644
index 00000000000..c6bcf4e681f
--- /dev/null
+++ b/t/helper/test-refs-advertise.c
@@ -0,0 +1,202 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *refs_advertise_usage[] = {
+	"test-tool refs-advertise [<options>...]",
+	NULL
+};
+
+static int can_upload_pack;
+static int can_receive_pack;
+static int can_ls_refs;
+static int die_read_version;
+static int die_write_version;
+static int die_read_first_ref;
+static int die_read_second_ref;
+static int die_filter_refs;
+static int upload_pack;
+static int receive_pack;
+static int ls_refs;
+static int verbose;
+static int version = 1;
+static int first_ref;
+static int second_ref;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list returns = STRING_LIST_INIT_NODUP;
+
+struct command {
+	struct command *next;
+	const char *error_string;
+	unsigned int skip_update:1,
+		     did_not_exist:1;
+	int index;
+	struct object_id old_oid;
+	struct object_id new_oid;
+	char ref_name[FLEX_ARRAY]; /* more */
+};
+
+static void refs_advertise_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	if (die_read_version)
+		die("die with the --die-read-version option");
+
+	for (;;) {
+		int linelen;
+
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		/* Ignore version negotiation for version 0 */
+		if (version == 0)
+			continue;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+			linelen = strlen(reader->line);
+			if (linelen < reader->pktlen) {
+				const char *feature_list = reader->line + linelen + 1;
+				if (parse_feature_request(feature_list, "git-upload-pack"))
+					upload_pack = 1;
+				if (parse_feature_request(feature_list, "git-receive-pack"))
+					receive_pack = 1;
+				if (parse_feature_request(feature_list, "ls-refs"))
+					ls_refs = 1;
+			}
+		}
+	}
+
+	if (die_write_version)
+		die("die with the --die-write-version option");
+
+	if (can_upload_pack || can_receive_pack || can_ls_refs)
+		packet_write_fmt(1, "version=%d%c%s%s%s\n",
+				version, '\0',
+				can_upload_pack ? "git-upload-pack ": "",
+				can_receive_pack? "git-receive-pack ": "",
+				can_ls_refs? "ls-refs ": "");
+	else
+		packet_write_fmt(1, "version=%d\n", version);
+
+	packet_flush(1);
+
+	if ((upload_pack && !can_upload_pack) ||
+		(receive_pack && !can_receive_pack) ||
+		(ls_refs && !can_ls_refs)) {
+			exit(0);
+	}
+}
+
+static void refs_advertise_read_refs(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+	int filter_ok = 0;
+
+	if (!first_ref) {
+		if (die_read_first_ref)
+			die("die with the --die-read-first-ref option");
+
+		first_ref = 1;
+	}
+
+	if (first_ref && !second_ref) {
+		if (die_read_second_ref)
+			die("die with the --die-read-second-ref option");
+
+		second_ref = 1;
+	}
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = buf.buf;
+
+	if (unsorted_string_list_has_string(&returns, p)) {
+		filter_ok = 1;
+	}
+
+	// if it's a ref filter request, we response without the commit id
+	if ((buf.len > (hash_size + 1)) && (strncmp("obj ", buf.buf, 4)))
+		strbuf_setlen(&buf, buf.len - (hash_size + 1));
+
+	if (filter_ok) {
+		packet_write_fmt(1, "%s %s\n", "ok", p);
+	} else {
+		packet_write_fmt(1, "%s %s\n", "ng", p);
+	}
+
+	if (die_filter_refs)
+		die("die with the --die-filter-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__refs_advertise(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "can-upload-pack", &can_upload_pack,
+			 "support upload-pack command"),
+		OPT_BOOL(0, "can-receive-pack", &can_receive_pack,
+			 "support upload-pack command"),
+		OPT_BOOL(0, "can-ls-refs", &can_ls_refs,
+			 "support ls-refs command"),
+		OPT_BOOL(0, "die-read-version", &die_read_version,
+			 "die when reading version"),
+		OPT_BOOL(0, "die-write-version", &die_write_version,
+			 "die when writing version"),
+		OPT_BOOL(0, "die-read-first-ref", &die_read_first_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
+			 "die when reading second reference"),
+		OPT_BOOL(0, "die-filter-refs", &die_filter_refs,
+			 "die when filtering refs"),
+		OPT_STRING_LIST('r', "return", &returns, "ref<SP>$refname<SP>$oid|obj<SP>$oid",
+				"refs or objects that can advertise"),
+		OPT__VERBOSE(&verbose, "be verbose"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, refs_advertise_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", refs_advertise_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	refs_advertise_verison(&reader);
+	for (;;) {
+		refs_advertise_read_refs(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 318fdbab0c3..10795e4eed6 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -69,6 +69,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "refs-advertise", cmd__refs_advertise },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index bb799271631..8f651e5cf9e 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -58,6 +58,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__refs_advertise(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/upload-pack.c b/upload-pack.c
index 09f48317b02..86714986d5f 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -27,6 +27,7 @@
 #include "commit-graph.h"
 #include "commit-reach.h"
 #include "shallow.h"
+#include "refs/refs-advertise.h"
 
 /* Remember to update object flag allocation in object.h */
 #define THEY_HAVE	(1u << 11)
@@ -1118,7 +1119,7 @@ static void receive_needs(struct upload_pack_data *data,
 		}
 
 		o = parse_object(the_repository, &oid_buf);
-		if (!o) {
+		if ((!o) || (filter_advertise_object(&oid_buf))) {
 			packet_writer_error(&data->writer,
 					    "upload-pack: not our ref %s",
 					    oid_to_hex(&oid_buf));
@@ -1164,6 +1165,12 @@ static int mark_our_ref(const char *refname, const char *refname_full,
 		o->flags |= HIDDEN_REF;
 		return 1;
 	}
+
+	if (filter_advertise_ref(refname, oid)) {
+		o->flags |= HIDDEN_REF;
+		return 1;
+	}
+
 	o->flags |= OUR_REF;
 	return 0;
 }
@@ -1183,8 +1190,10 @@ static void format_symref_info(struct strbuf *buf, struct string_list *symref)
 
 	if (!symref->nr)
 		return;
-	for_each_string_list_item(item, symref)
-		strbuf_addf(buf, " symref=%s:%s", item->string, (char *)item->util);
+	for_each_string_list_item(item, symref) {
+		if (!filter_advertise_ref((char *)item->util, NULL))
+			strbuf_addf(buf, " symref=%s:%s", item->string, (char *)item->util);
+	}
 }
 
 static void format_session_id(struct strbuf *buf, struct upload_pack_data *d) {
@@ -1198,11 +1207,19 @@ static int send_ref(const char *refname, const struct object_id *oid,
 	static const char *capabilities = "multi_ack thin-pack side-band"
 		" side-band-64k ofs-delta shallow deepen-since deepen-not"
 		" deepen-relative no-progress include-tag multi_ack_detailed";
-	const char *refname_nons = strip_namespace(refname);
 	struct object_id peeled;
 	struct upload_pack_data *data = cb_data;
+	const char *refname_nons = strip_namespace(refname);
+	const char *refname_to_filter = refname_nons;
 
-	if (mark_our_ref(refname_nons, refname, oid))
+	if (!strcmp(refname_nons, "HEAD")) {
+		struct string_list_item *item = string_list_lookup(&data->symref, "HEAD");
+		if (item) {
+			refname_to_filter = (const char *)item->util;
+		}
+	}
+
+	if (mark_our_ref(refname_to_filter, refname, oid))
 		return 0;
 
 	if (capabilities) {
@@ -1342,12 +1359,19 @@ static void get_upload_pack_config(struct upload_pack_data *data)
 	git_protected_config(upload_pack_protected_config, data);
 }
 
+static void clean_refs_advertise_filter(void) {
+	clean_advertise_refs_filter();
+}
+
 void upload_pack(const int advertise_refs, const int stateless_rpc,
 		 const int timeout)
 {
 	struct packet_reader reader;
 	struct upload_pack_data data;
 
+	create_advertise_refs_filter("git-upload-pack");
+	atexit(clean_refs_advertise_filter);
+
 	upload_pack_data_init(&data);
 	get_upload_pack_config(&data);
 
@@ -1421,7 +1445,7 @@ static int parse_want(struct packet_writer *writer, const char *line,
 		else
 			o = parse_object(the_repository, &oid);
 
-		if (!o) {
+		if ((!o) || (filter_advertise_object(&oid))) {
 			packet_writer_error(writer,
 					    "upload-pack: not our ref %s",
 					    oid_to_hex(&oid));
@@ -1453,7 +1477,8 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
 
 		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
 		if (ref_is_hidden(refname_nons, refname.buf) ||
-		    read_ref(refname.buf, &oid)) {
+		    read_ref(refname.buf, &oid) ||
+			filter_advertise_ref(refname_nons, &oid)) {
 			packet_writer_error(writer, "unknown ref %s", refname_nons);
 			die("unknown ref %s", refname_nons);
 		}
@@ -1701,6 +1726,9 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 
 	clear_object_flags(ALL_FLAGS);
 
+	create_advertise_refs_filter("git-upload-pack");
+	atexit(clean_refs_advertise_filter);
+
 	upload_pack_data_init(&data);
 	data.use_sideband = LARGE_PACKET_MAX;
 	get_upload_pack_config(&data);
-- 
gitgitgadget

[PATCH 2/3] t1419: add test cases for refs-advertise hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'refs-advertise' hook which is used to
filter the references during reference discovery phase and commit
fetching phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 t/t1419-refs-advertise-hooks.sh               | 138 ++++++++++
 t/t1419/common-functions.sh                   |  80 ++++++
 .../once-0000-abnormal-refs-advertise-hook.sh | 199 ++++++++++++++
 t/t1419/test-0000-standard-git-clone.sh       |  56 ++++
 t/t1419/test-0001-standard-git-push.sh        |  34 +++
 ...0002-ls-remote-with-refs-advertise-hook.sh |  95 +++++++
 ...03-upload-pack-with-refs-advertise-hook.sh | 127 +++++++++
 ...4-receive-pack-with-refs-advertise-hook.sh | 251 ++++++++++++++++++
 8 files changed, 980 insertions(+)
 create mode 100755 t/t1419-refs-advertise-hooks.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0000-standard-git-clone.sh
 create mode 100644 t/t1419/test-0001-standard-git-push.sh
 create mode 100644 t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh
 create mode 100644 t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh

diff --git a/t/t1419-refs-advertise-hooks.sh b/t/t1419-refs-advertise-hooks.sh
new file mode 100755
index 00000000000..4b4edb6267e
--- /dev/null
+++ b/t/t1419-refs-advertise-hooks.sh
@@ -0,0 +1,138 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test refs-advertise hook'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+setup_bare_repo_and_work_repo () {
+	# Refs of bare_repo : main(A)
+	# Refs of work_repo: main(A)  tags/v123
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git bare_repo.git.dump &&
+		rm -rf work_repo work_repo.dump &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git remote add origin ../bare_repo.git &&
+			git update-ref refs/heads/dev $A &&
+			git update-ref refs/heads/main $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push origin +refs/heads/*:refs/heads/* &&
+			git push origin +refs/tags/*:refs/tags/* &&
+			git push origin +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# setup pre-receive hook
+		write_script bare_repo.git/hooks/pre-receive <<-\EOF &&
+		exec >&2
+		echo "# pre-receive hook"
+		while read old new ref
+		do
+			echo "pre-receive< $old $new $ref"
+		done
+		EOF
+
+		# setup update hook
+		write_script bare_repo.git/hooks/update <<-\EOF &&
+		exec >&2
+		echo "# update hook"
+		echo "update< $@"
+		EOF
+
+		# setup post-receive hook
+		write_script bare_repo.git/hooks/post-receive <<-\EOF
+		exec >&2
+		echo "# post-receive hook"
+		while read old new ref
+		do
+			echo "post-receive< $old $new $ref"
+		done
+		EOF
+	'
+}
+
+run_refs_advertise_hook_tests() {
+	case $1 in
+		http)
+			PROTOCOL="HTTP protocol"
+			BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			BAREREPO_PREFIX="$HTTPD_URL"/smart
+			;;
+		local)
+			PROTOCOL="builtin protocol"
+			BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+			BAREREPO_PREFIX="$(pwd)"
+			;;
+	esac
+
+	BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+	GIT_TEST_PROTOCOL_VERSION=$2
+
+	# Run test cases for 'refs-advertise' hook
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		# Initialize the bare_repo repository and work_repo
+		setup_bare_repo_and_work_repo
+		git -C work_repo remote set-url origin "$BAREREPO_URL"
+		cp -rf work_repo work_repo.dump
+
+		git -C bare_repo.git config --local http.receivepack true
+		cp -rf bare_repo.git bare_repo.git.dump
+
+		if test "$1" = "http"; then
+			setup_askpass_helper
+			rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			mv bare_repo.git "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+		fi
+
+		. "$t"
+	done
+}
+
+
+setup_bare_repo_and_work_repo
+BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+BAREREPO_PREFIX="$(pwd)"
+BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+# Load test cases that only need to be executed once.
+for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
+do
+	. "$t"
+done
+
+for protocol in 1 2
+do
+	# Run test cases for 'refs-advertise' hook on local file protocol.
+	run_refs_advertise_hook_tests local $protocol
+done
+
+ROOT_PATH="$PWD"
+. "$TEST_DIRECTORY"/lib-gpg.sh
+. "$TEST_DIRECTORY"/lib-httpd.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
+
+start_httpd
+set_askpass user@host pass@host
+
+# Run test cases for 'refs-advertise' hook on HTTP protocol.
+for protocol in 1 2
+do
+	run_refs_advertise_hook_tests http $protocol
+done
+
+test_done
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..ab17c138411
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,80 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-push, git-show-ref and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.  We also
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$TRASH_DIRECTORY/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
+}
+
+filter_out_refs_advertise_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+filter_out_user_friendly_and_stable_output () {
+	make_user_friendly_and_stable_output |
+		sed -n ${1+"$@"}
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/once-0000-abnormal-refs-advertise-hook.sh b/t/t1419/once-0000-abnormal-refs-advertise-hook.sh
new file mode 100644
index 00000000000..6e4cbf95b82
--- /dev/null
+++ b/t/t1419/once-0000-abnormal-refs-advertise-hook.sh
@@ -0,0 +1,199 @@
+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook refs-advertise
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when write version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook refs-advertise
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read first filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read second filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-filter-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die while filtring refs" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-filter-refs option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook refs-advertise
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when write version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook refs-advertise
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read first filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read second filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise \
+			--can-upload-pack \
+			--can-receive-pack \
+			--can-ls-refs \
+			--die-filter-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die while filtring refs" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-filter-refs option
+		fatal: hook refs-advertise died abnormally
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0000-standard-git-clone.sh b/t/t1419/test-0000-standard-git-clone.sh
new file mode 100644
index 00000000000..8de3d2c3d0a
--- /dev/null
+++ b/t/t1419/test-0000-standard-git-clone.sh
@@ -0,0 +1,56 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
+	refs_num=$(cat out | wc -l) &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	refs_num=$(cat out | wc -l) &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): upload-pack" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	filter_out_refs_advertise_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B> HEAD
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): receive-pack" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION receive-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	filter_out_refs_advertise_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0001-standard-git-push.sh b/t/t1419/test-0001-standard-git-push.sh
new file mode 100644
index 00000000000..d6e977e899a
--- /dev/null
+++ b/t/t1419/test-0001-standard-git-push.sh
@@ -0,0 +1,34 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+		EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh b/t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh
new file mode 100644
index 00000000000..3cb4473a676
--- /dev/null
+++ b/t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh
@@ -0,0 +1,95 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while refs-advertise hook handle no command" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B" \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote branches" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs  \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote pull refs and tags" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh b/t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh
new file mode 100644
index 00000000000..6c6a9aa57aa
--- /dev/null
+++ b/t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh
@@ -0,0 +1,127 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while refs-advertise hook handle no command" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B" \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone all refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a not advertised commit" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION init local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git fetch "$BAREREPO_URL" $D
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-upload-pack \
+		--can-ls-refs \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone pull refs and tags" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): clone a not advertised branch" '
+	rm -rf local.git &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone "$BAREREPO_URL" local.git -b main &&
+	test ! -d local.git
+'
diff --git a/t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh b/t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh
new file mode 100644
index 00000000000..1dac755b36a
--- /dev/null
+++ b/t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh
@@ -0,0 +1,251 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+		test-tool refs-advertise
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook handle no command" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while refs-advertise hook handle no command" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while refs-advertise hook handle no command" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise nothing" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-receive-pack \
+		--can-ls-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise nothing" '
+	create_commits_in work_repo E &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <ZERO-OID> <COMMIT-E>        Z
+		remote: error: cannot lock ref "refs/heads/main": reference already exists        Z
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (failed to update ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise nothing" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise nothing" '
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		error: unable to delete "dev": remote ref does not exist
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-receive-pack \
+		--can-ls-refs \
+		-r "ref refs/heads/dev $A" \
+		-r "obj $A" \
+		-r "ref refs/heads/main $B" \
+		-r "obj $B" \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise all refs" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
+	test-tool refs-advertise \
+		--can-receive-pack \
+		--can-ls-refs \
+		-r "ref refs/pull-requests/1/head $C" \
+		-r "obj $C" \
+		-r "ref refs/tags/v123 $TAG" \
+		-r "obj $TAG" \
+		-r "obj $D"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise pull refs and tags" '
+	create_commits_in work_repo E &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <ZERO-OID> <COMMIT-E>        Z
+		remote: error: cannot lock ref "refs/heads/main": reference already exists        Z
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (failed to update ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise pull refs and tags" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise pull refs and tags" '
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		error: unable to delete "dev": remote ref does not exist
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH 3/3] doc: add documentation for the refs-advertise hook

[Sun Chao via GitGitGadget]

From: Sun Chao <16657101987@163.com>

"git upload-pack" or "git recevie-pack" can use "refs-advertise"
hook to filter the references and commits during reference discovery
phase and commit fetching phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 70 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..616d00a4477 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,76 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[refs-advertise]]
+refs-advertise
+~~~~~~~~~~~~
+
+This hook is invoked by 'git-receive-pack' and 'git-upload-pack' during the
+reference discovery phase and the commit fetching phase, each reference and
+commit to fetch will be filtered by this hook. The hook executes once with
+no arguments for each 'git-upload-pack' and 'git-receive-pack' process and
+if the exit status is non-zero, `git push` and `git fetch` will abort.
+
+Once the hook is invoked, a version number and server process name
+('git-upload-pack' or 'git-receive-pack' or 'ls-refs') should send to it in
+packet-line format, followed by a flush-pkt. The hook should response with
+its version number and process name list it support. If the list does not
+contains the server process name, the server will close the connection with
+the hook and keep going without the hook child process.
+
+During reference discovery phase, each reference will be filtered by this hook,
+In the following example, the letter 'G' stands for 'git-receive-pack' or
+'git-upload-pack' and the letter 'H' stands for this hook. The hook decides if
+the reference will be advertised or not, it sends result back with pkt-line format
+protocol, a response in "ok ref <ref>" format followed by a flush-pkt means
+the references "<ref>" can be advertised, and "ng ref <ref>" means not.
+
+    # Version negotiation
+    G: PKT-LINE(version=1\0git-upload-pack)
+    G: flush-pkt
+    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
+    H: flush-pkt
+
+    # Send reference filter request to hook
+    G: PKT-LINE(ref <ref> <oid>)
+    G: flush-pkt
+
+    # Receive result from the hook.
+    # Case 1: this reference is valid
+    H: PKT-LINE(ok ref <ref>)
+    H: flush-pkt
+    # Case 2: this reference is filtered out
+    H: PKT-LINE(ng ref <ref>)
+    H: flush-pkt
+
+During commit fetch phase of 'git-upload-pack' process, git client may send `want <oid>`
+requests and 'git-upload-pack' will send object filter requests to the hook to check if
+the object "<oid>" will be sent to the client or not. In the following example, the letter
+'G' stands for 'git-upload-pack' and the letter 'H' stands for this hook.
+
+The hook will decides if a commit will be sent to the client or not, it sends result with
+pkt-line format protocol to `git-upload-pack`, a response in "ok obj <oid>" format
+followed by a flush-pkt means the object "<oid>" can be sent to client, and "ng obj <oid>"
+means not.
+
+    # Version negotiation
+    G: PKT-LINE(version=1\0ls-refs)
+    G: flush-pkt
+    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
+    H: flush-pkt
+
+    # Send commit filter request to hook
+    G: PKT-LINE(obj <oid>)
+    G: flush-pkt
+
+    # Receive result from the hook.
+    # Case 1: this object is valid
+    H: PKT-LINE(ok obj <oid>)
+    H: flush-pkt
+    # Case 2: this object is filtered out
+    H: PKT-LINE(ng obj <oid>)
+    H: flush-pkt
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget

Re: [PATCH 0/3] refs-advertise: add hook to filter advertised refs

[Junio C Hamano]

"Sun Chao via GitGitGadget" <gitgitgadget@gmail.com> writes:

> Gerrit is implemented by JGit and is known as a centralized workflow system
> which supports reference-level access control for repository. If we choose
> to work in centralized workflow like what Gerrit provided, reference-level
> access control is needed and is possible if we add a reference advertise
> filter hook just like what Gerrit did.

It may be one starting point, but is it sufficient to call it
"possible"?  The server side needs to tighten "fetch by object name"
to refuse to serve objects that are not reachable from any of the
refs advertised to the client requesting them.  IIRC, fetch protocol
v2 is wide open and does not limit the requests to those that are
only reachable from the advertised refs.

Re: [PATCH 0/3] refs-advertise: add hook to filter advertised refs

[孙超]

> On Aug 4, 2022, at 04:27, Junio C Hamano <gitster@pobox.com> wrote:
> 
> "Sun Chao via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> Gerrit is implemented by JGit and is known as a centralized workflow system
>> which supports reference-level access control for repository. If we choose
>> to work in centralized workflow like what Gerrit provided, reference-level
>> access control is needed and is possible if we add a reference advertise
>> filter hook just like what Gerrit did.
> 
> It may be one starting point, but is it sufficient to call it
> "possible"?  The server side needs to tighten "fetch by object name"
> to refuse to serve objects that are not reachable from any of the
> refs advertised to the client requesting them.  IIRC, fetch protocol
> v2 is wide open and does not limit the requests to those that are
> only reachable from the advertised refs.
> 
> 
Hi Junio, thanks for you reply.

I agree with you that the server need to refuse the fetch requests that want to
steal objects not reachable from the advertised refs. So I have tried to understand
the source codes of of `ls-refs.c` and `upload-pack.c` (maybe I lost some important
files), and I put a new function `filter_advertise_object` to call `refs-advertise`
hook checks the `want` objects.

```diff
@@ -1118,7 +1119,7 @@ static void receive_needs(struct upload_pack_data *data,
               }

               o = parse_object(the_repository, &oid_buf);
-               if (!o) {
+               if ((!o) || (filter_advertise_object(&oid_buf))) {
                       packet_writer_error(&data->writer,
                                           "upload-pack: not our ref %s",
                                           oid_to_hex(&oid_buf));

...

@@ -1421,7 +1445,7 @@ static int parse_want(struct packet_writer *writer, const char *line,
               else
                       o = parse_object(the_repository, &oid);

-               if (!o) {
+               if ((!o) || (filter_advertise_object(&oid))) {
                       packet_writer_error(writer,
                                           "upload-pack: not our ref %s",
                                           oid_to_hex(&oid));
```

The `filter_advertise_object` will exchange messages with the hook by pkt-line
messages, eg:

       # Send commit filter request to hook
       G: PKT-LINE(obj <oid>)
       G: flush-pkt

       # Receive result from the hook.
       # Case 1: this object is valid
       H: PKT-LINE(ok obj <oid>)
       H: flush-pkt
       # Case 2: this object is filtered out
       H: PKT-LINE(ng obj <oid>)
       H: flush-pkt

the hook can check if the `oid` is valid for the client and returns `ng` message if not,
so git server will hide the objects to the client. And I added some test cases for
upload-pack V1 and V2 and looks like it works, but maybe I lost some important points and
I'm still trying to understand other codes for upload-pack and receive-pack because
currently I only implements the filter process for upload-pack and receive-pack.

Thanks for your reply again.

Re: [PATCH 0/3] refs-advertise: add hook to filter advertised refs

[Jiang Xin]

On Thu, Aug 4, 2022 at 12:31 AM Sun Chao via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> Gerrit is implemented by JGit and is known as a centralized workflow system
> which supports reference-level access control for repository. If we choose
> to work in centralized workflow like what Gerrit provided, reference-level
> access control is needed and is possible if we add a reference advertise
> filter hook just like what Gerrit did.
>
> This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
> during the reference discovery phase and the commit fetching phase, each
> reference and will be filtered by this hook. Git server can put
> reference-level control process to this hook and the git client does not
> need to change or known about that.

From the document you provided in patch 3/3, the hook returns not only
names of the references, but also OIDs. Since the oid of reference
should be provided as-is during the advertising phase, it is
sufficient for the hook to just return the visible reference names.

How about:
1. Implement a batch version of "ref_is_hidden()", such as
    "refs_batch_hidden()", to turn on or turn off the hidden bit
    for all references.

2. If there is an external hook, such as "hide-refs", call it instead
    of the config variables such as "transfer.hideRefs" to filter refs
    based on ACL and operations (read and write).

--
Jiang Xin

Re: [PATCH 0/3] refs-advertise: add hook to filter advertised refs

[孙超]

> On Aug 10, 2022, at 09:06, Jiang Xin <worldhello.net@gmail.com> wrote:
> 
> On Thu, Aug 4, 2022 at 12:31 AM Sun Chao via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> 
>> Gerrit is implemented by JGit and is known as a centralized workflow system
>> which supports reference-level access control for repository. If we choose
>> to work in centralized workflow like what Gerrit provided, reference-level
>> access control is needed and is possible if we add a reference advertise
>> filter hook just like what Gerrit did.
>> 
>> This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
>> during the reference discovery phase and the commit fetching phase, each
>> reference and will be filtered by this hook. Git server can put
>> reference-level control process to this hook and the git client does not
>> need to change or known about that.
> 
> From the document you provided in patch 3/3, the hook returns not only
> names of the references, but also OIDs. Since the oid of reference
> should be provided as-is during the advertising phase, it is
> sufficient for the hook to just return the visible reference names.
> 
> How about:
> 1. Implement a batch version of "ref_is_hidden()", such as
>    "refs_batch_hidden()", to turn on or turn off the hidden bit
>    for all references.
> 
> 2. If there is an external hook, such as "hide-refs", call it instead
>    of the config variables such as "transfer.hideRefs" to filter refs
>    based on ACL and operations (read and write).
> 
> --
> Jiang Xin
> 

Thanks a lot, Jiang Xin.

Your suggestion is right, for protocol V1 we do not need to filter the OIDs, and
there should be a configuration to turn on/off the hidden bit, I will try to add
such kind of configuration, maybe "transfer.hideRefs" is a good choice.

And after received Junio's reply I also did tests for V2, I find that even I
hide all the refs (by "git config transfer.hiderefs refs/" in upstream) the client
can still fetch specific object by it’s object id, here is the trace log:

```
.............................           trace: built-in: git fetch origin 5585e358b2a240ca8ed65a00008dbc865a1381c1
.............................           packet:        fetch< version 2
.............................           packet:        fetch< agent=git/2.37.1.288.gef002b009d
.............................           packet:        fetch> command=ls-refs
# the server does not advertise any refs
.............................           packet:        fetch< 0000
.............................           packet:        fetch> command=fetch
# the client send the want command with object oid
.............................           packet:        fetch> want 5585e358b2a240ca8ed65a00008dbc865a1381c1
.............................           packet:        fetch> done
.............................           packet:        fetch> 0000
.............................           packet:        fetch< packfile
# the client received the packfile contains the objects
.............................
From file:///local/upstream.git
 * branch                5585e358b2a240ca8ed65a00008dbc865a1381c1 -> FETCH_HEAD
```

Protocol V2 does not limit the request to the advertised refs, and if we want to
hide some refs, we need to hide the objects only reachable from them (for V2),
but it truly has performance issue for some huge repository.

[PATCH v2 0/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook hide-refs
to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference and will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
response with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" the references will hide to the
client and can not fetch its private data even in protocol V2.

    # Version negotiation
    G: PKT-LINE(version=1\0uploadpack)
    G: flush-pkt
    H: PKT-LINE(version=1)
    H: flush-pkt

    # Send reference filter request to hook
    G: PKT-LINE(ref <refname>:<refname_full>)
    G: flush-pkt

    # Receive result from the hook.
    # Case 1: this reference is hidden
    H: PKT-LINE(hide)
    H: flush-pkt

    # Case 2: this reference can be advertised
    H: flush-pkt


To enable the hide-refs hook, we should config hiderefs with force: option,
eg:

    git config --add transfer.hiderefs force:refs/prefix1/
    git config --add uploadpack.hiderefs force:!refs/prefix2/


the hide-refs will be called during reference discovery phase and check each
matched reference, a 'hide' response means the reference will be hidden for
its private data and even the allowTipSHA1InWant or allowReachableSHA1InWant
is set to true.

Sun Chao (3):
  hide-refs: add hook to force hide refs
  t1419: add test cases for hide-refs hook
  doc: add documentation for the hide-refs hook

 Documentation/githooks.txt                    |  48 ++++
 Makefile                                      |   1 +
 builtin/receive-pack.c                        |   5 +-
 ls-refs.c                                     |   2 +-
 refs.c                                        | 221 +++++++++++++++++-
 refs.h                                        |   6 +
 serve.c                                       |   2 +
 t/helper/test-hide-refs.c                     | 152 ++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 +++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 ++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 ++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 +++++++
 upload-pack.c                                 |  32 +--
 upload-pack.h                                 |   1 +
 18 files changed, 1111 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh


base-commit: afa70145a25e81faa685dc0b465e52b45d2444bd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v2
Pull-Request: https://github.com/git/git/pull/1301

Range-diff vs v1:

 1:  b4b5ce5a361 ! 1:  3b8fb63cc78 refs-advertise: add hook to filter advertised refs
     @@ Metadata
      Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
     -    refs-advertise: add hook to filter advertised refs
     +    hide-refs: add hook to force hide refs
      
          Gerrit is implemented by JGit and is known as a centralized workflow system
          which supports reference-level access control for repository. If we choose
          to work in centralized workflow like what Gerrit provided, reference-level
     -    access control is needed and is possible if we add a reference advertise
     -    filter hook just like what Gerrit did.
     +    access control is needed and we might add a reference filter hook
     +    `hide-refs` to hide the private data.
      
          This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
     -    during the reference discovery phase and the commit fetching phase,
     -    each reference and will be filtered by this hook. The hook executes once
     -    with no arguments for each 'git-upload-pack' and 'git-receive-pack' process
     -    and if the exit status is non-zero, `git push` and `git fetch` will abort.
     -
     -    Once the hook is invoked, a version number and server process name
     -    ('git-upload-pack' or 'git-receive-pack' or 'ls-refs') should send to it in
     -    packet-line format, followed by a flush-pkt. The hook should response with
     -    its version number and process name list it support. If the list does not
     -    contains the server process name, the server will close the connection with
     -    the hook and keep going without the hook child process.
     +    during the reference discovery phase, each reference and will be filtered
     +    with this hook. The hook executes once with no arguments for each
     +    'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
     +    a version number and server process name ('uploadpack' or 'receive') will
     +    send to it in pkt-line format, followed by a flush-pkt. The hook should
     +    response with its version number.
      
          During reference discovery phase, each reference will be filtered by this
          hook. In the following example, the letter 'G' stands for 'git-receive-pack'
          or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
     -    decides if the reference will be advertised or not, it sends result back in
     -    pkt-line format protocol, a response in "ok ref <ref>" format followed by a
     -    flush-pkt means the references "<ref>" can be advertised, and "ng ref <ref>"
     -    means not.
     -
     -        # Version negotiation
     -        G: PKT-LINE(version=1\0git-upload-pack)
     -        G: flush-pkt
     -        H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
     -        H: flush-pkt
     +    decides if the reference will be hidden or not, it sends result back in
     +    pkt-line format protocol, a response "hide" the references will hide
     +    to the client and can not fetch its private data even in protocol V2.
      
     -        # Send reference filter request to hook
     -        G: PKT-LINE(ref <ref> <oid>)
     -        G: flush-pkt
     +            # Version negotiation
     +            G: PKT-LINE(version=1\0uploadpack)
     +            G: flush-pkt
     +            H: PKT-LINE(version=1)
     +            H: flush-pkt
      
     -        # Receive result from the hook.
     -        # Case 1: this reference is valid
     -        H: PKT-LINE(ok ref <ref>)
     -        H: flush-pkt
     -        # Case 2: this reference is filtered out
     -        H: PKT-LINE(ng ref <ref>)
     -        H: flush-pkt
     +            # Send reference filter request to hook
     +            G: PKT-LINE(ref <refname>:<refname_full>)
     +            G: flush-pkt
      
     -    During commit fetch phase of 'git-upload-pack' process, git client may send
     -    `want <oid>` requests and 'git-upload-pack' will send object filter requests
     -    to the hook to check if the object "<oid>" will be sent to the client or
     -    not. In the following example, the letter 'G' stands for 'git-upload-pack'
     -    and the letter 'H' stands for this hook.
     +            # Receive result from the hook.
     +            # Case 1: this reference is hidden
     +            H: PKT-LINE(hide)
     +            H: flush-pkt
      
     -    The hook will decides if a commit will be sent to the client or not, it
     -    sends result in pkt-line format protocol to `git-upload-pack`, a response
     -    with "ok obj <oid>" format followed by a flush-pkt means the object "<oid>"
     -    can be sent to client, and "ng obj <oid>" means not.
     +            # Case 2: this reference can be advertised
     +            H: flush-pkt
      
     -        # Version negotiation
     -        G: PKT-LINE(version=1\0ls-refs)
     -        G: flush-pkt
     -        H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
     -        H: flush-pkt
     +    To enable the `hide-refs` hook, we should config hiderefs with `force:`
     +    option, eg:
      
     -        # Send commit filter request to hook
     -        G: PKT-LINE(obj <oid>)
     -        G: flush-pkt
     +            git config --add transfer.hiderefs force:refs/prefix1/
     +            git config --add uploadpack.hiderefs force:!refs/prefix2/
      
     -        # Receive result from the hook.
     -        # Case 1: this object is valid
     -        H: PKT-LINE(ok obj <oid>)
     -        H: flush-pkt
     -        # Case 2: this object is filtered out
     -        H: PKT-LINE(ng obj <oid>)
     -        H: flush-pkt
     +    the `hide-refs` will be called during reference discovery phase and
     +    check each matched reference, a 'hide' response means the reference will
     +    be hidden for its private data and even the `allowTipSHA1InWant` or
     +    `allowReachableSHA1InWant` is set to true.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ Makefile: TEST_BUILTINS_OBJS += test-wildmatch.o
       TEST_BUILTINS_OBJS += test-windows-named-pipe.o
       TEST_BUILTINS_OBJS += test-write-cache.o
       TEST_BUILTINS_OBJS += test-xml-encode.o
     -+TEST_BUILTINS_OBJS += test-refs-advertise.o
     ++TEST_BUILTINS_OBJS += test-hide-refs.o
       
       # Do not add more tests here unless they have extra dependencies. Add
       # them in TEST_BUILTINS_OBJS above.
     -@@ Makefile: LIB_OBJS += refs/files-backend.o
     - LIB_OBJS += refs/iterator.o
     - LIB_OBJS += refs/packed-backend.o
     - LIB_OBJS += refs/ref-cache.o
     -+LIB_OBJS += refs/refs-advertise.o
     - LIB_OBJS += refspec.o
     - LIB_OBJS += remote.o
     - LIB_OBJS += replace-object.o
      
       ## builtin/receive-pack.c ##
     -@@
     - #include "commit-reach.h"
     - #include "worktree.h"
     - #include "shallow.h"
     -+#include "refs/refs-advertise.h"
     - 
     - static const char * const receive_pack_usage[] = {
     - 	N_("git receive-pack <git-dir>"),
      @@ builtin/receive-pack.c: static int show_ref_cb(const char *path_full, const struct object_id *oid,
     - 	if (ref_is_hidden(path, path_full))
     + 	struct oidset *seen = data;
     + 	const char *path = strip_namespace(path_full);
     + 
     +-	if (ref_is_hidden(path, path_full))
     ++	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
       		return 0;
       
     -+	if (filter_advertise_ref(path, oid)) {
     -+		return 0;
     -+	}
     -+
       	/*
     - 	 * Advertise refs outside our current namespace as ".have"
     - 	 * refs, so that the client can use them to minimize data
     -@@ builtin/receive-pack.c: static int delete_only(struct command *commands)
     - 	return 1;
     - }
     - 
     -+static void clean_refs_advertise_filter(void) {
     -+	clean_advertise_refs_filter();
     -+}
     -+
     - int cmd_receive_pack(int argc, const char **argv, const char *prefix)
     - {
     - 	int advertise_refs = 0;
     -@@ builtin/receive-pack.c: int cmd_receive_pack(int argc, const char **argv, const char *prefix)
     - 	if (!enter_repo(service_dir, 0))
     - 		die("'%s' does not appear to be a git repository", service_dir);
     +@@ builtin/receive-pack.c: static void reject_updates_to_hidden(struct command *commands)
     + 		strbuf_setlen(&refname_full, prefix_len);
     + 		strbuf_addstr(&refname_full, cmd->ref_name);
       
     -+	create_advertise_refs_filter("git-receive-pack");
     -+	atexit(clean_refs_advertise_filter);
     -+
     - 	git_config(receive_pack_config, NULL);
     - 	if (cert_nonce_seed)
     - 		push_cert_nonce = prepare_push_cert_nonce(service_dir, time(NULL));
     +-		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
     ++		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
     ++			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
     + 			continue;
     + 		if (is_null_oid(&cmd->new_oid))
     + 			cmd->error_string = "deny deleting a hidden ref";
      
       ## ls-refs.c ##
     -@@
     - #include "ls-refs.h"
     - #include "pkt-line.h"
     - #include "config.h"
     -+#include "run-command.h"
     -+#include "refs/refs-advertise.h"
     - 
     - static int config_read;
     - static int advertise_unborn;
     - static int allow_unborn;
     -+static struct string_list symref = STRING_LIST_INIT_DUP;
     - 
     - static void ensure_config_read(void)
     - {
      @@ ls-refs.c: static int send_ref(const char *refname, const struct object_id *oid,
     - {
     - 	struct ls_refs_data *data = cb_data;
     - 	const char *refname_nons = strip_namespace(refname);
     -+	const char *refname_to_filter = refname_nons;
       
       	strbuf_reset(&data->buf);
       
     -@@ ls-refs.c: static int send_ref(const char *refname, const struct object_id *oid,
     - 	if (!ref_match(&data->prefixes, refname_nons))
     +-	if (ref_is_hidden(refname_nons, refname))
     ++	if (mark_our_ref(refname_nons, refname, oid))
       		return 0;
       
     -+	if (!strcmp(refname_nons, "HEAD")) {
     -+		struct string_list_item *item = string_list_lookup(&symref, "HEAD");
     -+		if (item) {
     -+			refname_to_filter = (const char *)item->util;
     -+		}
     -+	}
     -+
     -+	if (filter_advertise_ref(refname_to_filter, oid))
     -+		return 0;
     -+
     - 	if (oid)
     - 		strbuf_addf(&data->buf, "%s %s", oid_to_hex(oid), refname_nons);
     - 	else
     -@@ ls-refs.c: static int send_ref(const char *refname, const struct object_id *oid,
     - 
     - static void send_possibly_unborn_head(struct ls_refs_data *data)
     - {
     -+	const char *symref_target;
     -+	struct string_list_item *item;
     - 	struct strbuf namespaced = STRBUF_INIT;
     - 	struct object_id oid;
     - 	int flag;
     - 	int oid_is_null;
     - 
     - 	strbuf_addf(&namespaced, "%sHEAD", get_git_namespace());
     --	if (!resolve_ref_unsafe(namespaced.buf, 0, &oid, &flag))
     -+	symref_target = resolve_ref_unsafe(namespaced.buf, 0, &oid, &flag);
     -+	if (!symref_target)
     - 		return; /* bad ref */
     - 	oid_is_null = is_null_oid(&oid);
     - 	if (!oid_is_null ||
     --	    (data->unborn && data->symrefs && (flag & REF_ISSYMREF)))
     -+	    (data->unborn && data->symrefs && (flag & REF_ISSYMREF))) {
     -+		item = string_list_append(&symref, "HEAD");
     -+		item->util = xstrdup(strip_namespace(symref_target));
     - 		send_ref(namespaced.buf, oid_is_null ? NULL : &oid, flag, data);
     -+	}
     - 	strbuf_release(&namespaced);
     - }
     - 
     -@@ ls-refs.c: static int ls_refs_config(const char *var, const char *value, void *data)
     - 	return parse_hide_refs_config(var, value, "uploadpack");
     + 	if (!ref_match(&data->prefixes, refname_nons))
     +
     + ## refs.c ##
     +@@
     + #include "lockfile.h"
     + #include "iterator.h"
     + #include "refs.h"
     ++#include "pkt-line.h"
     + #include "refs/refs-internal.h"
     + #include "run-command.h"
     + #include "hook.h"
     +@@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
       }
       
     -+static void clean_refs_advertise_filter(void) {
     -+	clean_advertise_refs_filter();
     -+}
     -+
     - int ls_refs(struct repository *r, struct packet_reader *request)
     + static struct string_list *hide_refs;
     +-
     ++static struct string_list *force_hide_refs;
     ++static struct strbuf hide_refs_section = STRBUF_INIT;
     + int parse_hide_refs_config(const char *var, const char *value, const char *section)
       {
     - 	struct ls_refs_data data;
     -@@ ls-refs.c: int ls_refs(struct repository *r, struct packet_reader *request)
     + 	const char *key;
     ++	int force = 0;
     ++
     + 	if (!strcmp("transfer.hiderefs", var) ||
     + 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
     + 	     !strcmp(key, "hiderefs"))) {
     + 		char *ref;
     + 		int len;
     ++		int forcelen;
       
     - 	ensure_config_read();
     - 	git_config(ls_refs_config, NULL);
     -+	create_advertise_refs_filter("ls-refs");
     -+	atexit(clean_refs_advertise_filter);
     - 
     - 	while (packet_reader_read(request) == PACKET_READ_NORMAL) {
     - 		const char *arg = request->line;
     -
     - ## refs/refs-advertise.c (new) ##
     -@@
     -+#include "../cache.h"
     -+#include "../config.h"
     -+#include "../strbuf.h"
     -+#include "../hook.h"
     -+#include "../sigchain.h"
     -+#include "../pkt-line.h"
     -+#include "../refs.h"
     -+#include "../run-command.h"
     -+#include "connect.h"
     -+#include "ref-cache.h"
     -+#include "refs-advertise.h"
     + 		if (!value)
     + 			return config_error_nonbool(var);
      +
     -+struct advertise_refs_filter {
     -+	struct child_process proc;
     -+	struct packet_reader reader;
     -+};
     ++		forcelen = strlen("force:");
     ++		len = strlen(value);
     ++		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
     ++			if (len == forcelen)
     ++				return error(_("missing value for '%s' with force option"), var);
     ++
     ++			force = 1;
     ++			value += forcelen;
     ++		}
     ++
     + 		ref = xstrdup(value);
     + 		len = strlen(ref);
     + 		while (len && ref[len - 1] == '/')
     + 			ref[--len] = '\0';
     +-		if (!hide_refs) {
     +-			CALLOC_ARRAY(hide_refs, 1);
     +-			hide_refs->strdup_strings = 1;
     ++
     ++		if (force) {
     ++			if (!force_hide_refs) {
     ++				CALLOC_ARRAY(force_hide_refs, 1);
     ++				force_hide_refs->strdup_strings = 1;
     ++			}
     ++			string_list_append(force_hide_refs, ref);
     ++		} else {
     ++			if (!hide_refs) {
     ++				CALLOC_ARRAY(hide_refs, 1);
     ++				hide_refs->strdup_strings = 1;
     ++			}
     ++			string_list_append(hide_refs, ref);
     + 		}
     +-		string_list_append(hide_refs, ref);
     + 	}
      +
     -+static struct advertise_refs_filter *hook_filter = NULL;
     ++	if (hide_refs_section.len == 0) {
     ++		strbuf_addstr(&hide_refs_section, section);
     ++	}
      +
     -+void create_advertise_refs_filter(const char *command) {
     -+	struct advertise_refs_filter *filter;
     + 	return 0;
     + }
     + 
     +-int ref_is_hidden(const char *refname, const char *refname_full)
     ++static struct child_process *hide_refs_proc;
     ++static struct packet_reader *hide_refs_reader;
     ++static void create_hide_refs_process(void) {
      +	struct child_process *proc;
      +	struct packet_reader *reader;
      +	const char *hook_path;
     -+	int command_support = 0;
      +	int version = 0;
      +	int code;
      +
     -+	if (hook_filter != NULL)
     -+		return;
     -+
     -+	hook_path = find_hook("refs-advertise");
     ++	hook_path = find_hook("hide-refs");
      +	if (!hook_path) {
     -+		return;
     ++		die("can not find hide-refs hook");
      +	}
      +
     -+	filter = (struct advertise_refs_filter *) xcalloc (1, sizeof (struct advertise_refs_filter));
     -+	proc = &filter->proc;
     -+	reader = &filter->reader;
     ++	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
     ++	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
      +
      +	child_process_init(proc);
      +	strvec_push(&proc->args, hook_path);
      +	proc->in = -1;
      +	proc->out = -1;
     -+	proc->trace2_hook_name = "refs-advertise";
     ++	proc->trace2_hook_name = "hide-refs";
      +	proc->err = 0;
      +
      +	code = start_command(proc);
      +	if (code)
     -+		die("can not run hook refs-advertise");
     ++		die("can not run hook hide-refs");
      +
      +	sigchain_push(SIGPIPE, SIG_IGN);
      +
     @@ refs/refs-advertise.c (new)
      +	packet_reader_init(reader, proc->out, NULL, 0,
      +			   PACKET_READ_CHOMP_NEWLINE |
      +			   PACKET_READ_GENTLE_ON_EOF);
     -+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', command);
     ++	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
      +	if (!code)
      +		code = packet_flush_gently(proc->in);
      +
      +	if (!code)
      +		for (;;) {
     -+			int linelen;
      +			enum packet_read_status status;
      +
      +			status = packet_reader_read(reader);
      +			if (status != PACKET_READ_NORMAL) {
     -+				/* Check whether refs-advertise exited abnormally */
     ++				/* Check whether hide-refs exited abnormally */
      +				if (status == PACKET_READ_EOF)
     -+					die("can not read version message from hook refs-advertise");
     ++					die("can not read version message from hook hide-refs");
      +				break;
      +			}
      +
      +			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
      +				version = atoi(reader->line + 8);
     -+				linelen = strlen(reader->line);
     -+				if (linelen < reader->pktlen) {
     -+					const char *command_list = reader->line + linelen + 1;
     -+					if (parse_feature_request(command_list, command)) {
     -+						command_support = 1;
     -+					}
     -+				}
      +			}
      +		}
      +
      +	if (code)
     -+		die("can not read version message from hook refs-advertise");
     ++		die("can not read version message from hook hide-refs");
      +
      +	switch (version) {
      +	case 0:
     @@ refs/refs-advertise.c (new)
      +	case 1:
      +		break;
      +	default:
     -+		die(_("hook refs-advertise version '%d' is not supported"), version);
     ++		die(_("hook hide-refs version '%d' is not supported"), version);
      +	}
      +
      +	sigchain_pop(SIGPIPE);
      +
     -+	if (!command_support) {
     -+		close(proc->in);
     -+		close(proc->out);
     -+		kill(proc->pid, SIGTERM);
     -+		finish_command_in_signal(proc);
     -+		free(filter);
     -+
     -+		return;
     -+	}
     -+
     -+	hook_filter = filter;
     ++	hide_refs_proc = proc;
     ++	hide_refs_reader = reader;
      +	return;
      +}
      +
     -+void clean_advertise_refs_filter(void) {
     -+	struct child_process *proc;
     -+
     -+	if (!hook_filter) {
     -+		return;
     -+	}
     -+
     -+	proc = &hook_filter->proc;
     -+
     -+	close(proc->in);
     -+	close(proc->out);
     -+	kill(proc->pid, SIGTERM);
     -+	finish_command_in_signal(proc);
     -+	FREE_AND_NULL(hook_filter);
     -+}
     -+
     -+static int do_filter_advertise_ref(const char *refname, const struct object_id *oid) {
     -+	struct child_process *proc;
     -+	struct packet_reader *reader;
     ++static int ref_force_hidden_check(const char *refname, const char *refname_full)
     ++{
      +	struct strbuf buf = STRBUF_INIT;
     -+	char *oid_hex;
      +	int code;
     ++	int ret = 0;
     ++
     ++	if (!force_hide_refs) {
     ++		return 0;
     ++	}
      +
     -+	proc = &hook_filter->proc;
     -+	reader = &hook_filter->reader;
     -+	if (oid)
     -+		oid_hex = oid_to_hex(oid);
     -+	else
     -+		oid_hex = oid_to_hex(null_oid());
     ++	if (!hide_refs_proc) {
     ++		create_hide_refs_process();
     ++	}
      +
     -+	code = packet_write_fmt_gently(proc->in, "ref %s %s", refname, oid_hex);
     ++	sigchain_push(SIGPIPE, SIG_IGN);
     ++	code = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
      +	if (code)
     -+		die("hook refs-advertise died abnormally");
     ++		die("hook hide-refs died abnormally");
      +
     -+	code = packet_flush_gently(proc->in);
     ++	code = packet_flush_gently(hide_refs_proc->in);
      +	if (code)
     -+		die("hook refs-advertise died abnormally");
     ++		die("hook hide-refs died abnormally");
      +
      +	for (;;) {
      +		enum packet_read_status status;
      +
     -+		status = packet_reader_read(reader);
     ++		status = packet_reader_read(hide_refs_reader);
      +		if (status != PACKET_READ_NORMAL) {
     -+			/* Check whether refs-advertise exited abnormally */
     ++			/* Check whether hide-refs exited abnormally */
      +			if (status == PACKET_READ_EOF)
     -+				die("hook refs-advertise died abnormally");
     ++				die("hook hide-refs died abnormally");
      +			break;
      +		}
      +
      +		strbuf_reset(&buf);
     -+		strbuf_addstr(&buf, reader->line);
     ++		strbuf_addstr(&buf, hide_refs_reader->line);
      +	}
      +
     -+	if (strncmp("ok ref ", buf.buf, 7))
     -+		return -1;
     ++	if (!strncmp("hide", buf.buf, 4))
     ++		ret = 1;
      +
     -+	if (strcmp(refname, buf.buf + 7))
     -+		return -1;
     -+
     -+	return 0;
     -+}
     -+
     -+int filter_advertise_ref(const char *refname, const struct object_id *oid) {
     -+	int result;
     -+
     -+	if (!hook_filter) {
     -+		return 0;
     -+	}
     -+
     -+	sigchain_push(SIGPIPE, SIG_IGN);
     -+	result = do_filter_advertise_ref(refname, oid);
      +	sigchain_pop(SIGPIPE);
     -+
     -+	return result;
     ++	return ret;
      +}
      +
     -+static int do_filter_advertise_object(const struct object_id *oid) {
     -+	struct child_process *proc;
     -+	struct packet_reader *reader;
     -+	struct strbuf buf = STRBUF_INIT;
     -+	char *oid_hex;
     -+	int code;
     -+
     -+	proc = &hook_filter->proc;
     -+	reader = &hook_filter->reader;
     -+	oid_hex = oid_to_hex(oid);
     -+
     -+	code = packet_write_fmt_gently(proc->in, "obj %s", oid_hex);
     -+	if (code)
     -+		die("hook refs-advertise died abnormally");
     -+
     -+	code = packet_flush_gently(proc->in);
     -+	if (code)
     -+		die("hook refs-advertise died abnormally");
     -+
     -+	for (;;) {
     -+		enum packet_read_status status;
     ++static int ref_hidden_check(const char *refname, const char *refname_full, int force)
     + {
     ++	struct string_list *hide_refs_list = hide_refs;
     + 	int i;
     + 
     +-	if (!hide_refs)
     ++	if (force)
     ++		hide_refs_list = force_hide_refs;
      +
     -+		status = packet_reader_read(reader);
     -+		if (status != PACKET_READ_NORMAL) {
     -+			/* Check whether refs-advertise exited abnormally */
     -+			if (status == PACKET_READ_EOF)
     -+				die("hook refs-advertise died abnormally");
     -+			break;
     ++	if (!hide_refs_list)
     + 		return 0;
     +-	for (i = hide_refs->nr - 1; i >= 0; i--) {
     +-		const char *match = hide_refs->items[i].string;
     ++	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
     ++		const char *match = hide_refs_list->items[i].string;
     + 		const char *subject;
     + 		int neg = 0;
     + 		const char *p;
     +@@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
     + 		/* refname can be NULL when namespaces are used. */
     + 		if (subject &&
     + 		    skip_prefix(subject, match, &p) &&
     +-		    (!*p || *p == '/'))
     +-			return !neg;
     ++		    (!*p || *p == '/')) {
     ++			if (neg)
     ++				return 0;
     ++			if (!force)
     ++				return 1;
     ++			return ref_force_hidden_check(refname, refname_full);
      +		}
     + 	}
     + 	return 0;
     + }
     + 
     ++int ref_is_hidden(const char *refname, const char *refname_full)
     ++{
     ++	return ref_hidden_check(refname, refname_full, 0);
     ++}
      +
     -+		strbuf_reset(&buf);
     -+		strbuf_addstr(&buf, reader->line);
     -+	}
     -+
     -+	if (strncmp("ok obj ", buf.buf, 7))
     -+		return -1;
     -+
     -+	if (strcmp(oid_hex, buf.buf + 7))
     -+		return -1;
     -+
     -+	return 0;
     ++int ref_is_force_hidden(const char *refname, const char *refname_full)
     ++{
     ++	return ref_hidden_check(refname, refname_full, 1);
      +}
      +
     -+int filter_advertise_object(const struct object_id *oid) {
     -+	int result;
     ++#define OUR_REF		(1u << 12)
     ++#define HIDDEN_REF	(1u << 19)
     ++#define HIDDEN_REF_FORCE	(1u << 20)
     ++static int has_force_hidden;
     ++int mark_our_ref(const char *refname, const char *refname_full,
     ++			const struct object_id *oid)
     ++{
     ++	struct object *o;
      +
     -+	if (!hook_filter || !oid) {
     ++	if (!oid || is_null_oid(oid)) {
      +		return 0;
      +	}
      +
     -+	sigchain_push(SIGPIPE, SIG_IGN);
     -+	result = do_filter_advertise_object(oid);
     -+	sigchain_pop(SIGPIPE);
     ++	o = lookup_unknown_object(the_repository, oid);
     ++	if (ref_is_force_hidden(refname, refname_full)) {
     ++		o->flags |= HIDDEN_REF_FORCE;
     ++		has_force_hidden = 1;
     ++		return 1;
     ++	}
     ++	if (ref_is_hidden(refname, refname_full)) {
     ++		o->flags |= HIDDEN_REF;
     ++		return 1;
     ++	}
     ++	o->flags |= OUR_REF;
     ++	return 0;
     ++}
      +
     -+	return result;
     ++int has_force_hidden_refs(void) {
     ++	return has_force_hidden;
      +}
     ++
     + const char *find_descendant_ref(const char *dirname,
     + 				const struct string_list *extras,
     + 				const struct string_list *skip)
     +
     + ## refs.h ##
     +@@ refs.h: int parse_hide_refs_config(const char *var, const char *value, const char *);
     +  * parameter always points to the full ref name.
     +  */
     + int ref_is_hidden(const char *, const char *);
     ++int ref_is_force_hidden(const char *, const char *);
     ++/* return non-zero if the ref is hidden, otherwise 0 */
     ++int mark_our_ref(const char *refname, const char *refname_full,
     ++			const struct object_id *oid);
     ++int has_force_hidden_refs(void);
     ++void lazy_load_hidden_refs(void);
     + 
     + enum ref_type {
     + 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
      
     - ## refs/refs-advertise.h (new) ##
     + ## serve.c ##
      @@
     -+#ifndef REFS_REFS_ADVERTISE_H
     -+#define REFS_REFS_ADVERTISE_H
     -+
     -+#include "../hash.h"
     -+
     -+void create_advertise_refs_filter(const char *command);
     -+void clean_advertise_refs_filter(void);
     -+int filter_advertise_ref(const char *refname, const struct object_id *oid);
     -+int filter_advertise_object(const struct object_id *oid);
     -+
     -+#endif
     + #include "cache.h"
     + #include "repository.h"
     + #include "config.h"
     ++#include "refs.h"
     + #include "pkt-line.h"
     + #include "version.h"
     + #include "ls-refs.h"
     +@@ serve.c: void protocol_v2_serve_loop(int stateless_rpc)
     + 	 * a single request/response exchange
     + 	 */
     + 	if (stateless_rpc) {
     ++		lazy_load_hidden_refs();
     + 		process_request();
     + 	} else {
     + 		for (;;)
      
     - ## t/helper/test-refs-advertise.c (new) ##
     + ## t/helper/test-hide-refs.c (new) ##
      @@
      +#include "cache.h"
      +#include "hash.h"
     @@ t/helper/test-refs-advertise.c (new)
      +#include "sigchain.h"
      +#include "test-tool.h"
      +
     -+static const char *refs_advertise_usage[] = {
     -+	"test-tool refs-advertise [<options>...]",
     ++static const char *hide_refs_usage[] = {
     ++	"test-tool hide-refs [<options>...]",
      +	NULL
      +};
      +
     -+static int can_upload_pack;
     -+static int can_receive_pack;
     -+static int can_ls_refs;
      +static int die_read_version;
      +static int die_write_version;
      +static int die_read_first_ref;
      +static int die_read_second_ref;
     -+static int die_filter_refs;
     -+static int upload_pack;
     -+static int receive_pack;
     -+static int ls_refs;
     ++static int die_after_proc_ref;
      +static int verbose;
      +static int version = 1;
      +static int first_ref;
     @@ t/helper/test-refs-advertise.c (new)
      +	char ref_name[FLEX_ARRAY]; /* more */
      +};
      +
     -+static void refs_advertise_verison(struct packet_reader *reader) {
     ++static void hide_refs_verison(struct packet_reader *reader) {
      +	int server_version = 0;
      +
      +	if (die_read_version)
      +		die("die with the --die-read-version option");
      +
      +	for (;;) {
     -+		int linelen;
     -+
      +		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
      +			break;
      +
     @@ t/helper/test-refs-advertise.c (new)
      +			server_version = atoi(reader->line+8);
      +			if (server_version != 1)
      +				die("bad protocol version: %d", server_version);
     -+			linelen = strlen(reader->line);
     -+			if (linelen < reader->pktlen) {
     -+				const char *feature_list = reader->line + linelen + 1;
     -+				if (parse_feature_request(feature_list, "git-upload-pack"))
     -+					upload_pack = 1;
     -+				if (parse_feature_request(feature_list, "git-receive-pack"))
     -+					receive_pack = 1;
     -+				if (parse_feature_request(feature_list, "ls-refs"))
     -+					ls_refs = 1;
     -+			}
      +		}
      +	}
      +
      +	if (die_write_version)
      +		die("die with the --die-write-version option");
      +
     -+	if (can_upload_pack || can_receive_pack || can_ls_refs)
     -+		packet_write_fmt(1, "version=%d%c%s%s%s\n",
     -+				version, '\0',
     -+				can_upload_pack ? "git-upload-pack ": "",
     -+				can_receive_pack? "git-receive-pack ": "",
     -+				can_ls_refs? "ls-refs ": "");
     -+	else
     -+		packet_write_fmt(1, "version=%d\n", version);
     -+
     ++	packet_write_fmt(1, "version=%d\n", version);
      +	packet_flush(1);
     -+
     -+	if ((upload_pack && !can_upload_pack) ||
     -+		(receive_pack && !can_receive_pack) ||
     -+		(ls_refs && !can_ls_refs)) {
     -+			exit(0);
     -+	}
      +}
      +
     -+static void refs_advertise_read_refs(struct packet_reader *reader)
     ++static void hide_refs_proc(struct packet_reader *reader)
      +{
      +	const char *p;
      +	struct strbuf buf = STRBUF_INIT;
      +	enum packet_read_status status;
     -+	int filter_ok = 0;
      +
      +	if (!first_ref) {
      +		if (die_read_first_ref)
     @@ t/helper/test-refs-advertise.c (new)
      +		strbuf_addstr(&buf, reader->line);
      +	}
      +
     -+	p = buf.buf;
     -+
     -+	if (unsorted_string_list_has_string(&returns, p)) {
     -+		filter_ok = 1;
     -+	}
     -+
     -+	// if it's a ref filter request, we response without the commit id
     -+	if ((buf.len > (hash_size + 1)) && (strncmp("obj ", buf.buf, 4)))
     -+		strbuf_setlen(&buf, buf.len - (hash_size + 1));
     -+
     -+	if (filter_ok) {
     -+		packet_write_fmt(1, "%s %s\n", "ok", p);
     -+	} else {
     -+		packet_write_fmt(1, "%s %s\n", "ng", p);
     ++	p = strchr(buf.buf, ':');
     ++	if (unsorted_string_list_has_string(&returns, p + 1)) {
     ++		packet_write_fmt(1, "hide");
      +	}
      +
     -+	if (die_filter_refs)
     -+		die("die with the --die-filter-refs option");
     ++	if (die_after_proc_ref)
     ++		die("die with the --die-after-proc-refs option");
      +
      +	packet_flush(1);
      +}
      +
     -+int cmd__refs_advertise(int argc, const char **argv) {
     ++int cmd__hide_refs(int argc, const char **argv) {
      +	int nongit_ok = 0;
      +	struct packet_reader reader;
      +	const char *value = NULL;
      +	struct option options[] = {
     -+		OPT_BOOL(0, "can-upload-pack", &can_upload_pack,
     -+			 "support upload-pack command"),
     -+		OPT_BOOL(0, "can-receive-pack", &can_receive_pack,
     -+			 "support upload-pack command"),
     -+		OPT_BOOL(0, "can-ls-refs", &can_ls_refs,
     -+			 "support ls-refs command"),
      +		OPT_BOOL(0, "die-read-version", &die_read_version,
      +			 "die when reading version"),
      +		OPT_BOOL(0, "die-write-version", &die_write_version,
     @@ t/helper/test-refs-advertise.c (new)
      +			 "die when reading first reference"),
      +		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
      +			 "die when reading second reference"),
     -+		OPT_BOOL(0, "die-filter-refs", &die_filter_refs,
     -+			 "die when filtering refs"),
     -+		OPT_STRING_LIST('r', "return", &returns, "ref<SP>$refname<SP>$oid|obj<SP>$oid",
     -+				"refs or objects that can advertise"),
     ++		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
     ++			 "die after proc ref"),
     ++		OPT_STRING_LIST('r', "reserved", &returns, "refs-to-force-hidden",
     ++				"refs that will force hide"),
      +		OPT__VERBOSE(&verbose, "be verbose"),
      +		OPT_INTEGER('V', "version", &version,
      +			    "use this protocol version number"),
     @@ t/helper/test-refs-advertise.c (new)
      +
      +	setup_git_directory_gently(&nongit_ok);
      +
     -+	argc = parse_options(argc, argv, "test-tools", options, refs_advertise_usage, 0);
     ++	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
      +	if (argc > 0)
     -+		usage_msg_opt("Too many arguments.", refs_advertise_usage, options);
     ++		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
      +
      +	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
      +
     @@ t/helper/test-refs-advertise.c (new)
      +			hash_size = GIT_SHA256_HEXSZ;
      +	}
      +
     -+	refs_advertise_verison(&reader);
     ++	hide_refs_verison(&reader);
      +	for (;;) {
     -+		refs_advertise_read_refs(&reader);
     ++		hide_refs_proc(&reader);
      +	}
      +
      +	return 0;
     @@ t/helper/test-tool.c: static struct test_cmd cmds[] = {
       	{ "regex", cmd__regex },
       	{ "repository", cmd__repository },
       	{ "revision-walking", cmd__revision_walking },
     -+	{ "refs-advertise", cmd__refs_advertise },
     ++	{ "hide-refs", cmd__hide_refs },
       	{ "run-command", cmd__run_command },
       	{ "scrap-cache-tree", cmd__scrap_cache_tree },
       	{ "serve-v2", cmd__serve_v2 },
     @@ t/helper/test-tool.h: int cmd__reftable(int argc, const char **argv);
       int cmd__regex(int argc, const char **argv);
       int cmd__repository(int argc, const char **argv);
       int cmd__revision_walking(int argc, const char **argv);
     -+int cmd__refs_advertise(int argc, const char **argv);
     ++int cmd__hide_refs(int argc, const char **argv);
       int cmd__run_command(int argc, const char **argv);
       int cmd__scrap_cache_tree(int argc, const char **argv);
       int cmd__serve_v2(int argc, const char **argv);
      
       ## upload-pack.c ##
      @@
     - #include "commit-graph.h"
     - #include "commit-reach.h"
     - #include "shallow.h"
     -+#include "refs/refs-advertise.h"
     - 
     - /* Remember to update object flag allocation in object.h */
     - #define THEY_HAVE	(1u << 11)
     -@@ upload-pack.c: static void receive_needs(struct upload_pack_data *data,
     - 		}
     - 
     - 		o = parse_object(the_repository, &oid_buf);
     --		if (!o) {
     -+		if ((!o) || (filter_advertise_object(&oid_buf))) {
     - 			packet_writer_error(&data->writer,
     - 					    "upload-pack: not our ref %s",
     - 					    oid_to_hex(&oid_buf));
     -@@ upload-pack.c: static int mark_our_ref(const char *refname, const char *refname_full,
     - 		o->flags |= HIDDEN_REF;
     - 		return 1;
     - 	}
     -+
     -+	if (filter_advertise_ref(refname, oid)) {
     -+		o->flags |= HIDDEN_REF;
     -+		return 1;
     -+	}
     -+
     - 	o->flags |= OUR_REF;
     - 	return 0;
     - }
     -@@ upload-pack.c: static void format_symref_info(struct strbuf *buf, struct string_list *symref)
     + #define NOT_SHALLOW	(1u << 17)
     + #define CLIENT_SHALLOW	(1u << 18)
     + #define HIDDEN_REF	(1u << 19)
     ++#define HIDDEN_REF_FORCE	(1u << 20)
       
     - 	if (!symref->nr)
     - 		return;
     --	for_each_string_list_item(item, symref)
     --		strbuf_addf(buf, " symref=%s:%s", item->string, (char *)item->util);
     -+	for_each_string_list_item(item, symref) {
     -+		if (!filter_advertise_ref((char *)item->util, NULL))
     -+			strbuf_addf(buf, " symref=%s:%s", item->string, (char *)item->util);
     -+	}
     - }
     - 
     - static void format_session_id(struct strbuf *buf, struct upload_pack_data *d) {
     -@@ upload-pack.c: static int send_ref(const char *refname, const struct object_id *oid,
     - 	static const char *capabilities = "multi_ack thin-pack side-band"
     - 		" side-band-64k ofs-delta shallow deepen-since deepen-not"
     - 		" deepen-relative no-progress include-tag multi_ack_detailed";
     --	const char *refname_nons = strip_namespace(refname);
     - 	struct object_id peeled;
     - 	struct upload_pack_data *data = cb_data;
     -+	const char *refname_nons = strip_namespace(refname);
     -+	const char *refname_to_filter = refname_nons;
     - 
     --	if (mark_our_ref(refname_nons, refname, oid))
     -+	if (!strcmp(refname_nons, "HEAD")) {
     -+		struct string_list_item *item = string_list_lookup(&data->symref, "HEAD");
     -+		if (item) {
     -+			refname_to_filter = (const char *)item->util;
     -+		}
     -+	}
     -+
     -+	if (mark_our_ref(refname_to_filter, refname, oid))
     - 		return 0;
     +-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
     +-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
     ++#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
     ++		NOT_SHALLOW | CLIENT_SHALLOW)
       
     - 	if (capabilities) {
     -@@ upload-pack.c: static void get_upload_pack_config(struct upload_pack_data *data)
     - 	git_protected_config(upload_pack_protected_config, data);
     + /* Enum for allowed unadvertised object request (UOR) */
     + enum allow_uor {
     +@@ upload-pack.c: static void receive_needs(struct upload_pack_data *data,
     + 		packet_flush(1);
       }
       
     -+static void clean_refs_advertise_filter(void) {
     -+	clean_advertise_refs_filter();
     -+}
     -+
     - void upload_pack(const int advertise_refs, const int stateless_rpc,
     - 		 const int timeout)
     +-/* return non-zero if the ref is hidden, otherwise 0 */
     +-static int mark_our_ref(const char *refname, const char *refname_full,
     +-			const struct object_id *oid)
     +-{
     +-	struct object *o = lookup_unknown_object(the_repository, oid);
     +-
     +-	if (ref_is_hidden(refname, refname_full)) {
     +-		o->flags |= HIDDEN_REF;
     +-		return 1;
     +-	}
     +-	o->flags |= OUR_REF;
     +-	return 0;
     +-}
     +-
     + static int check_ref(const char *refname_full, const struct object_id *oid,
     + 		     int flag, void *cb_data)
       {
     - 	struct packet_reader reader;
     - 	struct upload_pack_data data;
     - 
     -+	create_advertise_refs_filter("git-upload-pack");
     -+	atexit(clean_refs_advertise_filter);
     -+
     - 	upload_pack_data_init(&data);
     - 	get_upload_pack_config(&data);
     - 
     -@@ upload-pack.c: static int parse_want(struct packet_writer *writer, const char *line,
     - 		else
     - 			o = parse_object(the_repository, &oid);
     - 
     --		if (!o) {
     -+		if ((!o) || (filter_advertise_object(&oid))) {
     - 			packet_writer_error(writer,
     - 					    "upload-pack: not our ref %s",
     - 					    oid_to_hex(&oid));
      @@ upload-pack.c: static int parse_want_ref(struct packet_writer *writer, const char *line,
       
       		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
       		if (ref_is_hidden(refname_nons, refname.buf) ||
     --		    read_ref(refname.buf, &oid)) {
     -+		    read_ref(refname.buf, &oid) ||
     -+			filter_advertise_ref(refname_nons, &oid)) {
     ++			ref_is_force_hidden(refname_nons, refname.buf) ||
     + 		    read_ref(refname.buf, &oid)) {
       			packet_writer_error(writer, "unknown ref %s", refname_nons);
       			die("unknown ref %s", refname_nons);
     - 		}
     +@@ upload-pack.c: enum fetch_state {
     + 	FETCH_DONE,
     + };
     + 
     ++static int lazy_load_hidden = 0;
     ++// lazy load hidden refs for protocol V2
     ++void lazy_load_hidden_refs(void) {
     ++	lazy_load_hidden = 1;
     ++}
     ++
     + int upload_pack_v2(struct repository *r, struct packet_reader *request)
     + {
     + 	enum fetch_state state = FETCH_PROCESS_ARGS;
      @@ upload-pack.c: int upload_pack_v2(struct repository *r, struct packet_reader *request)
     + 				state = FETCH_DONE;
     + 			break;
     + 		case FETCH_SEND_PACK:
     ++			if (lazy_load_hidden) {
     ++				head_ref_namespaced(check_ref, NULL);
     ++				for_each_namespaced_ref(check_ref, NULL);
     ++			}
     ++			if (has_force_hidden_refs())
     ++				check_non_tip(&data);
     + 			send_wanted_ref_info(&data);
     + 			send_shallow_info(&data);
       
     - 	clear_object_flags(ALL_FLAGS);
     +
     + ## upload-pack.h ##
     +@@ upload-pack.h: struct strbuf;
     + int upload_pack_advertise(struct repository *r,
     + 			  struct strbuf *value);
       
     -+	create_advertise_refs_filter("git-upload-pack");
     -+	atexit(clean_refs_advertise_filter);
     -+
     - 	upload_pack_data_init(&data);
     - 	data.use_sideband = LARGE_PACKET_MAX;
     - 	get_upload_pack_config(&data);
     ++void lazy_load_hidden_refs(void);
     + #endif /* UPLOAD_PACK_H */
 2:  88504b3a08a ! 2:  72333c12c3f t1419: add test cases for refs-advertise hook
     @@ Metadata
      Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
     -    t1419: add test cases for refs-advertise hook
     +    t1419: add test cases for hide-refs hook
      
     -    Add test cases for the new 'refs-advertise' hook which is used to
     -    filter the references during reference discovery phase and commit
     -    fetching phase.
     +    Add test cases for the new 'hide-refs' hook which is used to
     +    filter the references during reference discovery phase.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     - ## t/t1419-refs-advertise-hooks.sh (new) ##
     + ## t/t1419-hide-refs-hook.sh (new) ##
      @@
      +#!/bin/sh
      +#
      +# Copyright (c) 2022 Sun Chao
      +#
      +
     -+test_description='Test refs-advertise hook'
     ++test_description='Test hide-refs hook'
      +
      +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
      +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
     @@ t/t1419-refs-advertise-hooks.sh (new)
      +	'
      +}
      +
     -+run_refs_advertise_hook_tests() {
     ++run_hide_refs_hook_tests() {
      +	case $1 in
      +		http)
      +			PROTOCOL="HTTP protocol"
     @@ t/t1419-refs-advertise-hooks.sh (new)
      +
      +	GIT_TEST_PROTOCOL_VERSION=$2
      +
     -+	# Run test cases for 'refs-advertise' hook
     ++	# Run test cases for 'hide-refs' hook
      +	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
      +	do
      +		# Initialize the bare_repo repository and work_repo
     @@ t/t1419-refs-advertise-hooks.sh (new)
      +		cp -rf work_repo work_repo.dump
      +
      +		git -C bare_repo.git config --local http.receivepack true
     ++		git -C bare_repo.git config --add transfer.hiderefs force:HEAD
     ++		git -C bare_repo.git config --add transfer.hiderefs force:refs
      +		cp -rf bare_repo.git bare_repo.git.dump
      +
      +		if test "$1" = "http"; then
     @@ t/t1419-refs-advertise-hooks.sh (new)
      +# Load test cases that only need to be executed once.
      +for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
      +do
     ++	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:HEAD
     ++	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:refs
      +	. "$t"
      +done
      +
      +for protocol in 1 2
      +do
     -+	# Run test cases for 'refs-advertise' hook on local file protocol.
     -+	run_refs_advertise_hook_tests local $protocol
     ++	# Run test cases for 'hide-refs' hook on local file protocol.
     ++	run_hide_refs_hook_tests local $protocol
      +done
      +
      +ROOT_PATH="$PWD"
     @@ t/t1419-refs-advertise-hooks.sh (new)
      +start_httpd
      +set_askpass user@host pass@host
      +
     -+# Run test cases for 'refs-advertise' hook on HTTP protocol.
     ++# Run test cases for 'hide-refs' hook on HTTP protocol.
      +for protocol in 1 2
      +do
     -+	run_refs_advertise_hook_tests http $protocol
     ++	run_hide_refs_hook_tests http $protocol
      +done
      +
      +test_done
     @@ t/t1419/common-functions.sh (new)
      +		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
      +}
      +
     -+filter_out_refs_advertise_output() {
     ++filter_out_hide_refs_output() {
      +	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
      +}
      +
     @@ t/t1419/common-functions.sh (new)
      +	test_cmp show-ref.expect show-ref.filtered
      +}
      
     - ## t/t1419/once-0000-abnormal-refs-advertise-hook.sh (new) ##
     + ## t/t1419/once-0000-abnormal-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-version
     ++test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read version" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs \
     ++			--die-read-version \
     ++			-r refs/heads/main
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read version" '
     ++test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read version" '
      +	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-version option
     -+		fatal: can not read version message from hook refs-advertise
     ++		fatal: can not read version message from hook hide-refs
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when write version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-write-version
     ++test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when write version" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-write-version
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when write version" '
     ++test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when write version" '
      +	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-write-version option
     -+		fatal: can not read version message from hook refs-advertise
     ++		fatal: can not read version message from hook hide-refs
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read first filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-first-ref
     ++test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read first filter request" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-read-first-ref
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read first filter request" '
     ++test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
      +	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-first-ref option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die when read second filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-second-ref
     ++test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read second filter request" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-read-second-ref
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die when read second filter request" '
     ++test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
      +	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-second-ref option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): setup refs-advertise hook which die while filtring refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-filter-refs
     ++test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die while filtring refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-after-proc-refs
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while refs-advertise hook die while filtring refs" '
     ++test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
      +	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-filter-refs option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: die with the --die-after-proc-refs option
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-version
     ++test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read version" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-read-version
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read version" '
     ++test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read version" '
      +	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-version option
     -+		fatal: can not read version message from hook refs-advertise
     ++		fatal: can not read version message from hook hide-refs
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when write version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-write-version
     ++test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when write version" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-write-version
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when write version" '
     ++test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when write version" '
      +	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-write-version option
     -+		fatal: can not read version message from hook refs-advertise
     ++		fatal: can not read version message from hook hide-refs
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read first filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-first-ref
     ++test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read first filter request" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-read-first-ref
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read first filter request" '
     ++test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
      +	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-first-ref option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die when read second filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-read-second-ref
     ++test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read second filter request" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-read-second-ref
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die when read second filter request" '
     ++test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
      +	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
      +		fatal: die with the --die-read-second-ref option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): setup refs-advertise hook which die while filtring refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise \
     -+			--can-upload-pack \
     -+			--can-receive-pack \
     -+			--can-ls-refs \
     -+			--die-filter-refs
     ++test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die while filtring refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs --die-after-proc-refs
      +	EOF
      +'
      +
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while refs-advertise hook die while filtring refs" '
     ++test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
      +	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
      +	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-filter-refs option
     -+		fatal: hook refs-advertise died abnormally
     ++		fatal: die with the --die-after-proc-refs option
     ++		fatal: hook hide-refs died abnormally
      +	EOF
      +	test_cmp expect actual
      +'
      
     - ## t/t1419/test-0000-standard-git-clone.sh (new) ##
     + ## t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone" '
     -+	rm -rf local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
     -+	refs_num=$(cat out | wc -l) &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+		<COMMIT-D> refs/tags/v123^{}
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs
      +	EOF
     -+	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
     -+	refs_num=$(cat out | wc -l) &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
      +		<COMMIT-B>	HEAD
     @@ t/t1419/test-0000-standard-git-clone.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): upload-pack" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     -+	filter_out_refs_advertise_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-B> HEAD
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+		<COMMIT-D> refs/tags/v123^{}
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): receive-pack" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION receive-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     -+	filter_out_refs_advertise_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -
     - ## t/t1419/test-0001-standard-git-push.sh (new) ##
     -@@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main" '
     -+	create_commits_in work_repo E &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
     -+		To <URL/of/bare_repo.git>
     -+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		To <URL/of/bare_repo.git>
     -+		 - [deleted]         dev
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -
     - ## t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh (new) ##
     -@@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev" \
     ++		-r "refs/heads/main" \
     ++		-r "refs/pull-requests/1/head" \
     ++		-r "refs/tags/v123"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while refs-advertise hook handle no command" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide all refs" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		<COMMIT-B>	HEAD
     -+		<COMMIT-A>	refs/heads/dev
     -+		<COMMIT-B>	refs/heads/main
     -+		<COMMIT-C>	refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123>	refs/tags/v123
     -+		<COMMIT-D>	refs/tags/v123^{}
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B" \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $D"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev" \
     ++		-r "refs/heads/main"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote all refs" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide branches" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		<COMMIT-B>	HEAD
     -+		<COMMIT-A>	refs/heads/dev
     -+		<COMMIT-B>	refs/heads/main
      +		<COMMIT-C>	refs/pull-requests/1/head
      +		<COMMIT-TAG-v123>	refs/tags/v123
      +		<COMMIT-D>	refs/tags/v123^{}
     @@ t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "refs/pull-requests/1/head" \
     ++		-r "refs/tags/v123"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote branches" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide pull refs and tags" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     @@ t/t1419/test-0002-ls-remote-with-refs-advertise-hook.sh (new)
      +		<COMMIT-B>	refs/heads/main
      +	EOF
      +	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs  \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $D"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote pull refs and tags" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-C>	refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123>	refs/tags/v123
     -+		<COMMIT-D>	refs/tags/v123^{}
     -+	EOF
     -+	test_cmp expect actual
      +'
      
     - ## t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh (new) ##
     + ## t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while refs-advertise hook handle no command" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide no refs" '
      +	rm -rf local.git &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
     ++	git -C local.git show-ref -d >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
      +		<COMMIT-A> refs/heads/dev
     @@ t/t1419/test-0003-upload-pack-with-refs-advertise-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B" \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $TAG" \
     -+		-r "obj $D"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev" \
     ++		-r "refs/heads/main" \
     ++		-r "refs/pull-requests/1/head" \
     ++		-r "refs/tags/v123"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone all refs" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide all refs" '
      +	rm -rf local.git &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
     ++	test_must_fail git -C local.git show-ref -d >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+		<COMMIT-D> refs/tags/v123^{}
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev" \
     ++		-r "refs/heads/main"
      +	EOF
      +'
      +
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
      +	rm -rf local.git &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
     ++	git -C local.git show-ref -d >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     ++		<COMMIT-C> refs/pull-requests/1/head
     ++		<COMMIT-TAG-v123> refs/tags/v123
     ++		<COMMIT-D> refs/tags/v123^{}
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which some branches" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a not advertised commit" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a tip commit which is not hidden" '
      +	rm -rf local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION init local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git remote add origin "$BAREREPO_URL" &&
     -+	test_must_fail git -C local.git fetch "$BAREREPO_URL" $D
     ++	git init local.git &&
     ++	git -C local.git remote add origin "$BAREREPO_URL" &&
     ++	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
     ++'
     ++
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
     ++	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
     ++	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-upload-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $TAG" \
     -+		-r "obj $D"
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is not hidden" '
     ++	rm -rf local.git &&
     ++	git init local.git &&
     ++	git -C local.git remote add origin "$BAREREPO_URL" &&
     ++	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $A
     ++'
     ++
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "refs/pull-requests/1/head" \
     ++		-r "refs/tags/v123"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone pull refs and tags" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide pull refs and tags" '
      +	rm -rf local.git &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C local.git show-ref -d >out 2>&1 &&
     ++	git -C local.git show-ref -d >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+		<COMMIT-D> refs/tags/v123^{}
     ++		<COMMIT-A> refs/heads/dev
     ++		<COMMIT-B> refs/heads/main
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): clone a not advertised branch" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
     ++	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
     ++	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
     ++'
     ++
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is hidden" '
      +	rm -rf local.git &&
     -+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone "$BAREREPO_URL" local.git -b main &&
     -+	test ! -d local.git
     ++	git init local.git &&
     ++	git -C local.git remote add origin "$BAREREPO_URL" &&
     ++	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $C
      +'
      
     - ## t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh (new) ##
     + ## t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which handle no command" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+		test-tool refs-advertise
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++		test-tool hide-refs
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook handle no command" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide no refs" '
      +	create_commits_in work_repo E &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
     @@ t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while refs-advertise hook handle no command" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while hide-refs hook hide no refs" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
      +	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     @@ t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while refs-advertise hook handle no command" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while hide-refs hook hide no refs" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
      +	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     @@ t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise nothing" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
      +	rm -rf work_repo &&
      +	cp -rf work_repo.dump work_repo &&
      +	rm -rf "$BAREREPO_GIT_DIR" &&
      +	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-receive-pack \
     -+		--can-ls-refs
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	test-tool hide-refs \
     ++		-r "HEAD" \
     ++		-r "refs/heads/dev" \
     ++		-r "refs/heads/main" \
     ++		-r "refs/pull-requests/1/head" \
     ++		-r "refs/tags/v123"
      +	EOF
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise nothing" '
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide all refs" '
      +	create_commits_in work_repo E &&
      +	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
     @@ t/t1419/test-0004-receive-pack-with-refs-advertise-hook.sh (new)
      +	format_and_save_expect <<-EOF &&
      +		remote: # pre-receive hook        Z
      +		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/main <ZERO-OID> <COMMIT-E>        Z
     -+		remote: error: cannot lock ref "refs/heads/main": reference already exists        Z
      +		To <URL/of/bare_repo.git>
     -+		 ! [remote rejected] HEAD -> main (failed to update ref)
     -+		error: failed to push some refs to "<URL/of/bare_repo.git>"
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise nothing" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		To <URL/of/bare_repo.git>
     -+		 * [new branch]      HEAD -> new
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise nothing" '
     -+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		error: unable to delete "dev": remote ref does not exist
     -+		error: failed to push some refs to "<URL/of/bare_repo.git>"
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise all refs" '
     -+	rm -rf work_repo &&
     -+	cp -rf work_repo.dump work_repo &&
     -+	rm -rf "$BAREREPO_GIT_DIR" &&
     -+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-receive-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/heads/dev $A" \
     -+		-r "obj $A" \
     -+		-r "ref refs/heads/main $B" \
     -+		-r "obj $B" \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $TAG" \
     -+		-r "obj $D"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise all refs" '
     -+	create_commits_in work_repo E &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
     -+		To <URL/of/bare_repo.git>
     -+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise all refs" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		To <URL/of/bare_repo.git>
     -+		 * [new branch]      HEAD -> new
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise all refs" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		To <URL/of/bare_repo.git>
     -+		 - [deleted]         dev
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup refs-advertise hook which advertise pull refs and tags" '
     -+	rm -rf work_repo &&
     -+	cp -rf work_repo.dump work_repo &&
     -+	rm -rf "$BAREREPO_GIT_DIR" &&
     -+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
     -+	write_script "$BAREREPO_GIT_DIR/hooks/refs-advertise" <<-EOF
     -+	test-tool refs-advertise \
     -+		--can-receive-pack \
     -+		--can-ls-refs \
     -+		-r "ref refs/pull-requests/1/head $C" \
     -+		-r "obj $C" \
     -+		-r "ref refs/tags/v123 $TAG" \
     -+		-r "obj $TAG" \
     -+		-r "obj $D"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while refs-advertise hook advertise pull refs and tags" '
     -+	create_commits_in work_repo E &&
     -+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/main <ZERO-OID> <COMMIT-E>        Z
     -+		remote: error: cannot lock ref "refs/heads/main": reference already exists        Z
     -+		To <URL/of/bare_repo.git>
     -+		 ! [remote rejected] HEAD -> main (failed to update ref)
     -+		error: failed to push some refs to "<URL/of/bare_repo.git>"
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while while refs-advertise hook advertise pull refs and tags" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		To <URL/of/bare_repo.git>
     -+		 * [new branch]      HEAD -> new
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while while refs-advertise hook advertise pull refs and tags" '
     -+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		error: unable to delete "dev": remote ref does not exist
     ++		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
      +		error: failed to push some refs to "<URL/of/bare_repo.git>"
      +	EOF
      +	test_cmp expect actual
 3:  d98357d5f64 ! 3:  e737997eb31 doc: add documentation for the refs-advertise hook
     @@ Metadata
      Author: Sun Chao <16657101987@163.com>
      
       ## Commit message ##
     -    doc: add documentation for the refs-advertise hook
     +    doc: add documentation for the hide-refs hook
      
     -    "git upload-pack" or "git recevie-pack" can use "refs-advertise"
     -    hook to filter the references and commits during reference discovery
     -    phase and commit fetching phase.
     +    "git upload-pack" or "git recevie-pack" can use "hide-refs"
     +    hook to filter the references during reference discovery phase.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
       pushing anything.  Information about why the push is rejected may be sent
       to the user by writing to standard error.
       
     -+[[refs-advertise]]
     -+refs-advertise
     -+~~~~~~~~~~~~
     ++[[hide-refs]]
     ++hide-refs
     ++~~~~~~~~~
      +
     -+This hook is invoked by 'git-receive-pack' and 'git-upload-pack' during the
     -+reference discovery phase and the commit fetching phase, each reference and
     -+commit to fetch will be filtered by this hook. The hook executes once with
     -+no arguments for each 'git-upload-pack' and 'git-receive-pack' process and
     -+if the exit status is non-zero, `git push` and `git fetch` will abort.
     ++This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
     ++during the reference discovery phase, each reference and will be filtered
     ++by this hook. The hook executes once with no arguments for each
     ++'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
     ++a version number and server process name ('uploadpack' or 'receive') will
     ++send to it in pkt-line format, followed by a flush-pkt. The hook should
     ++response with its version number.
      +
     -+Once the hook is invoked, a version number and server process name
     -+('git-upload-pack' or 'git-receive-pack' or 'ls-refs') should send to it in
     -+packet-line format, followed by a flush-pkt. The hook should response with
     -+its version number and process name list it support. If the list does not
     -+contains the server process name, the server will close the connection with
     -+the hook and keep going without the hook child process.
     ++During reference discovery phase, each reference will be filtered by this
     ++hook. In the following example, the letter 'G' stands for 'git-receive-pack'
     ++or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
     ++decides if the reference will be hidden or not, it sends result back in
     ++pkt-line format protocol, a response "hide" the references will hide
     ++to the client and can not fetch it even in protocol V2.
      +
     -+During reference discovery phase, each reference will be filtered by this hook,
     -+In the following example, the letter 'G' stands for 'git-receive-pack' or
     -+'git-upload-pack' and the letter 'H' stands for this hook. The hook decides if
     -+the reference will be advertised or not, it sends result back with pkt-line format
     -+protocol, a response in "ok ref <ref>" format followed by a flush-pkt means
     -+the references "<ref>" can be advertised, and "ng ref <ref>" means not.
     ++	# Version negotiation
     ++	G: PKT-LINE(version=1\0uploadpack)
     ++	G: flush-pkt
     ++	H: PKT-LINE(version=1)
     ++	H: flush-pkt
      +
     -+    # Version negotiation
     -+    G: PKT-LINE(version=1\0git-upload-pack)
     -+    G: flush-pkt
     -+    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
     -+    H: flush-pkt
     ++	# Send reference filter request to hook
     ++	G: PKT-LINE(ref <refname>:<refnamefull>)
     ++	G: flush-pkt
      +
     -+    # Send reference filter request to hook
     -+    G: PKT-LINE(ref <ref> <oid>)
     -+    G: flush-pkt
     ++	# Receive result from the hook.
     ++	# Case 1: this reference is hidden
     ++	H: PKT-LINE(hide)
     ++	H: flush-pkt
      +
     -+    # Receive result from the hook.
     -+    # Case 1: this reference is valid
     -+    H: PKT-LINE(ok ref <ref>)
     -+    H: flush-pkt
     -+    # Case 2: this reference is filtered out
     -+    H: PKT-LINE(ng ref <ref>)
     -+    H: flush-pkt
     ++	# Case 2: this reference can be advertised
     ++	H: flush-pkt
      +
     -+During commit fetch phase of 'git-upload-pack' process, git client may send `want <oid>`
     -+requests and 'git-upload-pack' will send object filter requests to the hook to check if
     -+the object "<oid>" will be sent to the client or not. In the following example, the letter
     -+'G' stands for 'git-upload-pack' and the letter 'H' stands for this hook.
     ++To enable the `hide-refs` hook, we should config hiderefs with `force:`
     ++option, eg:
      +
     -+The hook will decides if a commit will be sent to the client or not, it sends result with
     -+pkt-line format protocol to `git-upload-pack`, a response in "ok obj <oid>" format
     -+followed by a flush-pkt means the object "<oid>" can be sent to client, and "ng obj <oid>"
     -+means not.
     ++	git config --add transfer.hiderefs force:refs/prefix1/
     ++	git config --add uploadpack.hiderefs force:!refs/prefix2/
      +
     -+    # Version negotiation
     -+    G: PKT-LINE(version=1\0ls-refs)
     -+    G: flush-pkt
     -+    H: PKT-LINE(version=1\0git-upload-pack git-receive-pack ls-refs)
     -+    H: flush-pkt
     -+
     -+    # Send commit filter request to hook
     -+    G: PKT-LINE(obj <oid>)
     -+    G: flush-pkt
     -+
     -+    # Receive result from the hook.
     -+    # Case 1: this object is valid
     -+    H: PKT-LINE(ok obj <oid>)
     -+    H: flush-pkt
     -+    # Case 2: this object is filtered out
     -+    H: PKT-LINE(ng obj <oid>)
     -+    H: flush-pkt
     ++the `hide-refs` will be called during reference discovery phase and
     ++check each matched reference, a 'hide' reponse means the reference will
     ++be hidden for its private data and even the `allowTipSHA1InWant` and
     ++`allowReachableSHA1InWant` is set to true.
      +
       [[pre-receive]]
       pre-receive

-- 
gitgitgadget

[PATCH v2 1/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook
`hide-refs` to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference and will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
response with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" the references will hide
to the client and can not fetch its private data even in protocol V2.

        # Version negotiation
        G: PKT-LINE(version=1\0uploadpack)
        G: flush-pkt
        H: PKT-LINE(version=1)
        H: flush-pkt

        # Send reference filter request to hook
        G: PKT-LINE(ref <refname>:<refname_full>)
        G: flush-pkt

        # Receive result from the hook.
        # Case 1: this reference is hidden
        H: PKT-LINE(hide)
        H: flush-pkt

        # Case 2: this reference can be advertised
        H: flush-pkt

To enable the `hide-refs` hook, we should config hiderefs with `force:`
option, eg:

        git config --add transfer.hiderefs force:refs/prefix1/
        git config --add uploadpack.hiderefs force:!refs/prefix2/

the `hide-refs` will be called during reference discovery phase and
check each matched reference, a 'hide' response means the reference will
be hidden for its private data and even the `allowTipSHA1InWant` or
`allowReachableSHA1InWant` is set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                  |   1 +
 builtin/receive-pack.c    |   5 +-
 ls-refs.c                 |   2 +-
 refs.c                    | 221 ++++++++++++++++++++++++++++++++++++--
 refs.h                    |   6 ++
 serve.c                   |   2 +
 t/helper/test-hide-refs.c | 152 ++++++++++++++++++++++++++
 t/helper/test-tool.c      |   1 +
 t/helper/test-tool.h      |   1 +
 upload-pack.c             |  32 +++---
 upload-pack.h             |   1 +
 11 files changed, 394 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c

diff --git a/Makefile b/Makefile
index 2ec9b2dc6bb..0c1865370cd 100644
--- a/Makefile
+++ b/Makefile
@@ -793,6 +793,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-hide-refs.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31b48e728be..16f2a21e97a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -296,7 +296,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,
 	struct oidset *seen = data;
 	const char *path = strip_namespace(path_full);
 
-	if (ref_is_hidden(path, path_full))
+	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
 		return 0;
 
 	/*
@@ -1794,7 +1794,8 @@ static void reject_updates_to_hidden(struct command *commands)
 		strbuf_setlen(&refname_full, prefix_len);
 		strbuf_addstr(&refname_full, cmd->ref_name);
 
-		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
+		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
+			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
 			continue;
 		if (is_null_oid(&cmd->new_oid))
 			cmd->error_string = "deny deleting a hidden ref";
diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..b5cb1316d38 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 	strbuf_reset(&data->buf);
 
-	if (ref_is_hidden(refname_nons, refname))
+	if (mark_our_ref(refname_nons, refname, oid))
 		return 0;
 
 	if (!ref_match(&data->prefixes, refname_nons))
diff --git a/refs.c b/refs.c
index 90bcb271687..4a6abbfe4fd 100644
--- a/refs.c
+++ b/refs.c
@@ -8,6 +8,7 @@
 #include "lockfile.h"
 #include "iterator.h"
 #include "refs.h"
+#include "pkt-line.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
 #include "hook.h"
@@ -1296,39 +1297,191 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 }
 
 static struct string_list *hide_refs;
-
+static struct string_list *force_hide_refs;
+static struct strbuf hide_refs_section = STRBUF_INIT;
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
 	const char *key;
+	int force = 0;
+
 	if (!strcmp("transfer.hiderefs", var) ||
 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
 	     !strcmp(key, "hiderefs"))) {
 		char *ref;
 		int len;
+		int forcelen;
 
 		if (!value)
 			return config_error_nonbool(var);
+
+		forcelen = strlen("force:");
+		len = strlen(value);
+		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
+			if (len == forcelen)
+				return error(_("missing value for '%s' with force option"), var);
+
+			force = 1;
+			value += forcelen;
+		}
+
 		ref = xstrdup(value);
 		len = strlen(ref);
 		while (len && ref[len - 1] == '/')
 			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
+
+		if (force) {
+			if (!force_hide_refs) {
+				CALLOC_ARRAY(force_hide_refs, 1);
+				force_hide_refs->strdup_strings = 1;
+			}
+			string_list_append(force_hide_refs, ref);
+		} else {
+			if (!hide_refs) {
+				CALLOC_ARRAY(hide_refs, 1);
+				hide_refs->strdup_strings = 1;
+			}
+			string_list_append(hide_refs, ref);
 		}
-		string_list_append(hide_refs, ref);
 	}
+
+	if (hide_refs_section.len == 0) {
+		strbuf_addstr(&hide_refs_section, section);
+	}
+
 	return 0;
 }
 
-int ref_is_hidden(const char *refname, const char *refname_full)
+static struct child_process *hide_refs_proc;
+static struct packet_reader *hide_refs_reader;
+static void create_hide_refs_process(void) {
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int version = 0;
+	int code;
+
+	hook_path = find_hook("hide-refs");
+	if (!hook_path) {
+		die("can not find hide-refs hook");
+	}
+
+	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
+	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "hide-refs";
+	proc->err = 0;
+
+	code = start_command(proc);
+	if (code)
+		die("can not run hook hide-refs");
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE |
+			   PACKET_READ_GENTLE_ON_EOF);
+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
+	if (!code)
+		code = packet_flush_gently(proc->in);
+
+	if (!code)
+		for (;;) {
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether hide-refs exited abnormally */
+				if (status == PACKET_READ_EOF)
+					die("can not read version message from hook hide-refs");
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+			}
+		}
+
+	if (code)
+		die("can not read version message from hook hide-refs");
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		die(_("hook hide-refs version '%d' is not supported"), version);
+	}
+
+	sigchain_pop(SIGPIPE);
+
+	hide_refs_proc = proc;
+	hide_refs_reader = reader;
+	return;
+}
+
+static int ref_force_hidden_check(const char *refname, const char *refname_full)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int code;
+	int ret = 0;
+
+	if (!force_hide_refs) {
+		return 0;
+	}
+
+	if (!hide_refs_proc) {
+		create_hide_refs_process();
+	}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	code = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	code = packet_flush_gently(hide_refs_proc->in);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(hide_refs_reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether hide-refs exited abnormally */
+			if (status == PACKET_READ_EOF)
+				die("hook hide-refs died abnormally");
+			break;
+		}
+
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, hide_refs_reader->line);
+	}
+
+	if (!strncmp("hide", buf.buf, 4))
+		ret = 1;
+
+	sigchain_pop(SIGPIPE);
+	return ret;
+}
+
+static int ref_hidden_check(const char *refname, const char *refname_full, int force)
 {
+	struct string_list *hide_refs_list = hide_refs;
 	int i;
 
-	if (!hide_refs)
+	if (force)
+		hide_refs_list = force_hide_refs;
+
+	if (!hide_refs_list)
 		return 0;
-	for (i = hide_refs->nr - 1; i >= 0; i--) {
-		const char *match = hide_refs->items[i].string;
+	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
+		const char *match = hide_refs_list->items[i].string;
 		const char *subject;
 		int neg = 0;
 		const char *p;
@@ -1348,12 +1501,58 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 		/* refname can be NULL when namespaces are used. */
 		if (subject &&
 		    skip_prefix(subject, match, &p) &&
-		    (!*p || *p == '/'))
-			return !neg;
+		    (!*p || *p == '/')) {
+			if (neg)
+				return 0;
+			if (!force)
+				return 1;
+			return ref_force_hidden_check(refname, refname_full);
+		}
 	}
 	return 0;
 }
 
+int ref_is_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 0);
+}
+
+int ref_is_force_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 1);
+}
+
+#define OUR_REF		(1u << 12)
+#define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
+static int has_force_hidden;
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid)
+{
+	struct object *o;
+
+	if (!oid || is_null_oid(oid)) {
+		return 0;
+	}
+
+	o = lookup_unknown_object(the_repository, oid);
+	if (ref_is_force_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF_FORCE;
+		has_force_hidden = 1;
+		return 1;
+	}
+	if (ref_is_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF;
+		return 1;
+	}
+	o->flags |= OUR_REF;
+	return 0;
+}
+
+int has_force_hidden_refs(void) {
+	return has_force_hidden;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
diff --git a/refs.h b/refs.h
index 47cb9edbaa8..ae584a644fa 100644
--- a/refs.h
+++ b/refs.h
@@ -818,6 +818,12 @@ int parse_hide_refs_config(const char *var, const char *value, const char *);
  * parameter always points to the full ref name.
  */
 int ref_is_hidden(const char *, const char *);
+int ref_is_force_hidden(const char *, const char *);
+/* return non-zero if the ref is hidden, otherwise 0 */
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid);
+int has_force_hidden_refs(void);
+void lazy_load_hidden_refs(void);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/serve.c b/serve.c
index 733347f602a..6b556719d9f 100644
--- a/serve.c
+++ b/serve.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "refs.h"
 #include "pkt-line.h"
 #include "version.h"
 #include "ls-refs.h"
@@ -320,6 +321,7 @@ void protocol_v2_serve_loop(int stateless_rpc)
 	 * a single request/response exchange
 	 */
 	if (stateless_rpc) {
+		lazy_load_hidden_refs();
 		process_request();
 	} else {
 		for (;;)
diff --git a/t/helper/test-hide-refs.c b/t/helper/test-hide-refs.c
new file mode 100644
index 00000000000..8dcd6700782
--- /dev/null
+++ b/t/helper/test-hide-refs.c
@@ -0,0 +1,152 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *hide_refs_usage[] = {
+	"test-tool hide-refs [<options>...]",
+	NULL
+};
+
+static int die_read_version;
+static int die_write_version;
+static int die_read_first_ref;
+static int die_read_second_ref;
+static int die_after_proc_ref;
+static int verbose;
+static int version = 1;
+static int first_ref;
+static int second_ref;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list returns = STRING_LIST_INIT_NODUP;
+
+struct command {
+	struct command *next;
+	const char *error_string;
+	unsigned int skip_update:1,
+		     did_not_exist:1;
+	int index;
+	struct object_id old_oid;
+	struct object_id new_oid;
+	char ref_name[FLEX_ARRAY]; /* more */
+};
+
+static void hide_refs_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	if (die_read_version)
+		die("die with the --die-read-version option");
+
+	for (;;) {
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		/* Ignore version negotiation for version 0 */
+		if (version == 0)
+			continue;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+		}
+	}
+
+	if (die_write_version)
+		die("die with the --die-write-version option");
+
+	packet_write_fmt(1, "version=%d\n", version);
+	packet_flush(1);
+}
+
+static void hide_refs_proc(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+
+	if (!first_ref) {
+		if (die_read_first_ref)
+			die("die with the --die-read-first-ref option");
+
+		first_ref = 1;
+	}
+
+	if (first_ref && !second_ref) {
+		if (die_read_second_ref)
+			die("die with the --die-read-second-ref option");
+
+		second_ref = 1;
+	}
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = strchr(buf.buf, ':');
+	if (unsorted_string_list_has_string(&returns, p + 1)) {
+		packet_write_fmt(1, "hide");
+	}
+
+	if (die_after_proc_ref)
+		die("die with the --die-after-proc-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__hide_refs(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "die-read-version", &die_read_version,
+			 "die when reading version"),
+		OPT_BOOL(0, "die-write-version", &die_write_version,
+			 "die when writing version"),
+		OPT_BOOL(0, "die-read-first-ref", &die_read_first_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
+			 "die when reading second reference"),
+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
+			 "die after proc ref"),
+		OPT_STRING_LIST('r', "reserved", &returns, "refs-to-force-hidden",
+				"refs that will force hide"),
+		OPT__VERBOSE(&verbose, "be verbose"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	hide_refs_verison(&reader);
+	for (;;) {
+		hide_refs_proc(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 318fdbab0c3..0c30be9fe44 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -69,6 +69,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "hide-refs", cmd__hide_refs },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index bb799271631..79e9068cbeb 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -58,6 +58,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__hide_refs(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/upload-pack.c b/upload-pack.c
index b217a1f469e..095d6c4cb40 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -38,9 +38,10 @@
 #define NOT_SHALLOW	(1u << 17)
 #define CLIENT_SHALLOW	(1u << 18)
 #define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
 
-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
+		NOT_SHALLOW | CLIENT_SHALLOW)
 
 /* Enum for allowed unadvertised object request (UOR) */
 enum allow_uor {
@@ -1155,20 +1156,6 @@ static void receive_needs(struct upload_pack_data *data,
 		packet_flush(1);
 }
 
-/* return non-zero if the ref is hidden, otherwise 0 */
-static int mark_our_ref(const char *refname, const char *refname_full,
-			const struct object_id *oid)
-{
-	struct object *o = lookup_unknown_object(the_repository, oid);
-
-	if (ref_is_hidden(refname, refname_full)) {
-		o->flags |= HIDDEN_REF;
-		return 1;
-	}
-	o->flags |= OUR_REF;
-	return 0;
-}
-
 static int check_ref(const char *refname_full, const struct object_id *oid,
 		     int flag, void *cb_data)
 {
@@ -1454,6 +1441,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
 
 		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
 		if (ref_is_hidden(refname_nons, refname.buf) ||
+			ref_is_force_hidden(refname_nons, refname.buf) ||
 		    read_ref(refname.buf, &oid)) {
 			packet_writer_error(writer, "unknown ref %s", refname_nons);
 			die("unknown ref %s", refname_nons);
@@ -1695,6 +1683,12 @@ enum fetch_state {
 	FETCH_DONE,
 };
 
+static int lazy_load_hidden = 0;
+// lazy load hidden refs for protocol V2
+void lazy_load_hidden_refs(void) {
+	lazy_load_hidden = 1;
+}
+
 int upload_pack_v2(struct repository *r, struct packet_reader *request)
 {
 	enum fetch_state state = FETCH_PROCESS_ARGS;
@@ -1740,6 +1734,12 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 				state = FETCH_DONE;
 			break;
 		case FETCH_SEND_PACK:
+			if (lazy_load_hidden) {
+				head_ref_namespaced(check_ref, NULL);
+				for_each_namespaced_ref(check_ref, NULL);
+			}
+			if (has_force_hidden_refs())
+				check_non_tip(&data);
 			send_wanted_ref_info(&data);
 			send_shallow_info(&data);
 
diff --git a/upload-pack.h b/upload-pack.h
index d6ee25ea98e..541610ba800 100644
--- a/upload-pack.h
+++ b/upload-pack.h
@@ -12,4 +12,5 @@ struct strbuf;
 int upload_pack_advertise(struct repository *r,
 			  struct strbuf *value);
 
+void lazy_load_hidden_refs(void);
 #endif /* UPLOAD_PACK_H */
-- 
gitgitgadget

[PATCH v2 2/3] t1419: add test cases for hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'hide-refs' hook which is used to
filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 ++++++++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 +++++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 +++++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 ++++++++++
 6 files changed, 669 insertions(+)
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh

diff --git a/t/t1419-hide-refs-hook.sh b/t/t1419-hide-refs-hook.sh
new file mode 100755
index 00000000000..52ce61c0e1f
--- /dev/null
+++ b/t/t1419-hide-refs-hook.sh
@@ -0,0 +1,142 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test hide-refs hook'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+setup_bare_repo_and_work_repo () {
+	# Refs of bare_repo : main(A)
+	# Refs of work_repo: main(A)  tags/v123
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git bare_repo.git.dump &&
+		rm -rf work_repo work_repo.dump &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git remote add origin ../bare_repo.git &&
+			git update-ref refs/heads/dev $A &&
+			git update-ref refs/heads/main $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push origin +refs/heads/*:refs/heads/* &&
+			git push origin +refs/tags/*:refs/tags/* &&
+			git push origin +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# setup pre-receive hook
+		write_script bare_repo.git/hooks/pre-receive <<-\EOF &&
+		exec >&2
+		echo "# pre-receive hook"
+		while read old new ref
+		do
+			echo "pre-receive< $old $new $ref"
+		done
+		EOF
+
+		# setup update hook
+		write_script bare_repo.git/hooks/update <<-\EOF &&
+		exec >&2
+		echo "# update hook"
+		echo "update< $@"
+		EOF
+
+		# setup post-receive hook
+		write_script bare_repo.git/hooks/post-receive <<-\EOF
+		exec >&2
+		echo "# post-receive hook"
+		while read old new ref
+		do
+			echo "post-receive< $old $new $ref"
+		done
+		EOF
+	'
+}
+
+run_hide_refs_hook_tests() {
+	case $1 in
+		http)
+			PROTOCOL="HTTP protocol"
+			BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			BAREREPO_PREFIX="$HTTPD_URL"/smart
+			;;
+		local)
+			PROTOCOL="builtin protocol"
+			BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+			BAREREPO_PREFIX="$(pwd)"
+			;;
+	esac
+
+	BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+	GIT_TEST_PROTOCOL_VERSION=$2
+
+	# Run test cases for 'hide-refs' hook
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		# Initialize the bare_repo repository and work_repo
+		setup_bare_repo_and_work_repo
+		git -C work_repo remote set-url origin "$BAREREPO_URL"
+		cp -rf work_repo work_repo.dump
+
+		git -C bare_repo.git config --local http.receivepack true
+		git -C bare_repo.git config --add transfer.hiderefs force:HEAD
+		git -C bare_repo.git config --add transfer.hiderefs force:refs
+		cp -rf bare_repo.git bare_repo.git.dump
+
+		if test "$1" = "http"; then
+			setup_askpass_helper
+			rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			mv bare_repo.git "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+		fi
+
+		. "$t"
+	done
+}
+
+
+setup_bare_repo_and_work_repo
+BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+BAREREPO_PREFIX="$(pwd)"
+BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+# Load test cases that only need to be executed once.
+for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
+do
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:HEAD
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:refs
+	. "$t"
+done
+
+for protocol in 1 2
+do
+	# Run test cases for 'hide-refs' hook on local file protocol.
+	run_hide_refs_hook_tests local $protocol
+done
+
+ROOT_PATH="$PWD"
+. "$TEST_DIRECTORY"/lib-gpg.sh
+. "$TEST_DIRECTORY"/lib-httpd.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
+
+start_httpd
+set_askpass user@host pass@host
+
+# Run test cases for 'hide-refs' hook on HTTP protocol.
+for protocol in 1 2
+do
+	run_hide_refs_hook_tests http $protocol
+done
+
+test_done
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..e86c3d11317
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,80 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-push, git-show-ref and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.  We also
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$TRASH_DIRECTORY/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
+}
+
+filter_out_hide_refs_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+filter_out_user_friendly_and_stable_output () {
+	make_user_friendly_and_stable_output |
+		sed -n ${1+"$@"}
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/once-0000-abnormal-hide-refs-hook.sh b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
new file mode 100644
index 00000000000..c4ff642264e
--- /dev/null
+++ b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
@@ -0,0 +1,161 @@
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs \
+			--die-read-version \
+			-r refs/heads/main
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..1766d869d58
--- /dev/null
+++ b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
@@ -0,0 +1,77 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide branches" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide pull refs and tags" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..d2fc5f0c4f4
--- /dev/null
+++ b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
@@ -0,0 +1,122 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide no refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide all refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	test_must_fail git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which some branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $A
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide pull refs and tags" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $C
+'
diff --git a/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..198fbf5fa5c
--- /dev/null
+++ b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
@@ -0,0 +1,87 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide no refs" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide all refs" '
+	create_commits_in work_repo E &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH v2 3/3] doc: add documentation for the hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <16657101987@163.com>

"git upload-pack" or "git recevie-pack" can use "hide-refs"
hook to filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..df712903464 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[hide-refs]]
+hide-refs
+~~~~~~~~~
+
+This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
+during the reference discovery phase, each reference and will be filtered
+by this hook. The hook executes once with no arguments for each
+'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
+a version number and server process name ('uploadpack' or 'receive') will
+send to it in pkt-line format, followed by a flush-pkt. The hook should
+response with its version number.
+
+During reference discovery phase, each reference will be filtered by this
+hook. In the following example, the letter 'G' stands for 'git-receive-pack'
+or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
+decides if the reference will be hidden or not, it sends result back in
+pkt-line format protocol, a response "hide" the references will hide
+to the client and can not fetch it even in protocol V2.
+
+	# Version negotiation
+	G: PKT-LINE(version=1\0uploadpack)
+	G: flush-pkt
+	H: PKT-LINE(version=1)
+	H: flush-pkt
+
+	# Send reference filter request to hook
+	G: PKT-LINE(ref <refname>:<refnamefull>)
+	G: flush-pkt
+
+	# Receive result from the hook.
+	# Case 1: this reference is hidden
+	H: PKT-LINE(hide)
+	H: flush-pkt
+
+	# Case 2: this reference can be advertised
+	H: flush-pkt
+
+To enable the `hide-refs` hook, we should config hiderefs with `force:`
+option, eg:
+
+	git config --add transfer.hiderefs force:refs/prefix1/
+	git config --add uploadpack.hiderefs force:!refs/prefix2/
+
+the `hide-refs` will be called during reference discovery phase and
+check each matched reference, a 'hide' reponse means the reference will
+be hidden for its private data and even the `allowTipSHA1InWant` and
+`allowReachableSHA1InWant` is set to true.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget

Re: [PATCH v2 3/3] doc: add documentation for the hide-refs hook

[Eric Sunshine]

On Sun, Aug 14, 2022 at 8:56 PM Sun Chao via GitGitGadget
<gitgitgadget@gmail.com> wrote:
> "git upload-pack" or "git recevie-pack" can use "hide-refs"

s/recevie/receive/

> hook to filter the references during reference discovery phase.
>
> Signed-off-by: Sun Chao <sunchao9@huawei.com>
> ---
> diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
> @@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
> +This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
> +during the reference discovery phase, each reference and will be filtered

s/and//

> +by this hook. The hook executes once with no arguments for each
> +'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
> +a version number and server process name ('uploadpack' or 'receive') will
> +send to it in pkt-line format, followed by a flush-pkt. The hook should
> +response with its version number.

s/response/respond/

> +During reference discovery phase, each reference will be filtered by this
> +hook. In the following example, the letter 'G' stands for 'git-receive-pack'
> +or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
> +decides if the reference will be hidden or not, it sends result back in
> +pkt-line format protocol, a response "hide" the references will hide
> +to the client and can not fetch it even in protocol V2.
> +
> +       # Version negotiation
> +       G: PKT-LINE(version=1\0uploadpack)
> +       G: flush-pkt
> +       H: PKT-LINE(version=1)
> +       H: flush-pkt
> +
> +       # Send reference filter request to hook
> +       G: PKT-LINE(ref <refname>:<refnamefull>)
> +       G: flush-pkt
> +
> +       # Receive result from the hook.
> +       # Case 1: this reference is hidden
> +       H: PKT-LINE(hide)
> +       H: flush-pkt
> +
> +       # Case 2: this reference can be advertised
> +       H: flush-pkt
> +
> +To enable the `hide-refs` hook, we should config hiderefs with `force:`
> +option, eg:
> +
> +       git config --add transfer.hiderefs force:refs/prefix1/
> +       git config --add uploadpack.hiderefs force:!refs/prefix2/
> +
> +the `hide-refs` will be called during reference discovery phase and
> +check each matched reference, a 'hide' reponse means the reference will

s/reponse/response/

> +be hidden for its private data and even the `allowTipSHA1InWant` and

s/and even the/even if/

> +`allowReachableSHA1InWant` is set to true.

s/is/are/

Re: [PATCH v2 3/3] doc: add documentation for the hide-refs hook

[孙超]

> On Aug 15, 2022, at 12:12, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> On Sun, Aug 14, 2022 at 8:56 PM Sun Chao via GitGitGadget
> <gitgitgadget@gmail.com> wrote:
>> "git upload-pack" or "git recevie-pack" can use "hide-refs"
> 
> s/recevie/receive/
> 
>> hook to filter the references during reference discovery phase.
>> 
>> Signed-off-by: Sun Chao <sunchao9@huawei.com>
>> ---
>> diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
>> @@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
>> +This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
>> +during the reference discovery phase, each reference and will be filtered
> 
> s/and//
> 
>> +by this hook. The hook executes once with no arguments for each
>> +'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
>> +a version number and server process name ('uploadpack' or 'receive') will
>> +send to it in pkt-line format, followed by a flush-pkt. The hook should
>> +response with its version number.
> 
> s/response/respond/
> 
>> +During reference discovery phase, each reference will be filtered by this
>> +hook. In the following example, the letter 'G' stands for 'git-receive-pack'
>> +or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
>> +decides if the reference will be hidden or not, it sends result back in
>> +pkt-line format protocol, a response "hide" the references will hide
>> +to the client and can not fetch it even in protocol V2.
>> +
>> +       # Version negotiation
>> +       G: PKT-LINE(version=1\0uploadpack)
>> +       G: flush-pkt
>> +       H: PKT-LINE(version=1)
>> +       H: flush-pkt
>> +
>> +       # Send reference filter request to hook
>> +       G: PKT-LINE(ref <refname>:<refnamefull>)
>> +       G: flush-pkt
>> +
>> +       # Receive result from the hook.
>> +       # Case 1: this reference is hidden
>> +       H: PKT-LINE(hide)
>> +       H: flush-pkt
>> +
>> +       # Case 2: this reference can be advertised
>> +       H: flush-pkt
>> +
>> +To enable the `hide-refs` hook, we should config hiderefs with `force:`
>> +option, eg:
>> +
>> +       git config --add transfer.hiderefs force:refs/prefix1/
>> +       git config --add uploadpack.hiderefs force:!refs/prefix2/
>> +
>> +the `hide-refs` will be called during reference discovery phase and
>> +check each matched reference, a 'hide' reponse means the reference will
> 
> s/reponse/response/
> 
>> +be hidden for its private data and even the `allowTipSHA1InWant` and
> 
> s/and even the/even if/
> 
>> +`allowReachableSHA1InWant` is set to true.
> 
> s/is/are/
> 

thanks a lot ! I will update the patches right now.

Re: [PATCH v2 3/3] doc: add documentation for the hide-refs hook

[Junio C Hamano]

孙超 <16657101987@163.com> writes:

> thanks a lot ! I will update the patches right now.

Not "right now".

Instead of flooding the list with repeated "oops that was wrong"
updates, it may be more effective use of others' time to wait for
more feedback before acting on them, and to take time to proofread
the result of your updates before sending them out.

Thanks.

[PATCH v3 0/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook hide-refs
to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered with
this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client and can not fetch its private data even in protocol V2.

            # Version negotiation
            G: PKT-LINE(version=1\0uploadpack)
            G: flush-pkt
            H: PKT-LINE(version=1)
            H: flush-pkt

            # Send reference filter request to hook
            G: PKT-LINE(ref <refname>:<refname_full>)
            G: flush-pkt

            # Receive result from the hook.
            # Case 1: this reference is hidden
            H: PKT-LINE(hide)
            H: flush-pkt

            # Case 2: this reference can be advertised
            H: flush-pkt


To enable the hide-refs hook, we should config hiderefs with force: option,
eg:

            git config --add transfer.hiderefs force:refs/prefix1/
            git config --add uploadpack.hiderefs force:!refs/prefix2/


the hide-refs will be called during reference discovery phase and check each
matched reference, a 'hide' response means the reference will be hidden for
its private data even if allowTipSHA1InWant or allowReachableSHA1InWant are
set to true.

Sun Chao (3):
  hide-refs: add hook to force hide refs
  t1419: add test cases for hide-refs hook
  doc: add documentation for the hide-refs hook

 Documentation/githooks.txt                    |  48 ++++
 Makefile                                      |   1 +
 builtin/receive-pack.c                        |   5 +-
 ls-refs.c                                     |   2 +-
 refs.c                                        | 221 +++++++++++++++++-
 refs.h                                        |   6 +
 serve.c                                       |   2 +
 t/helper/test-hide-refs.c                     | 152 ++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 +++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 ++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 ++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 +++++++
 upload-pack.c                                 |  32 +--
 upload-pack.h                                 |   1 +
 18 files changed, 1111 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh


base-commit: afa70145a25e81faa685dc0b465e52b45d2444bd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v3
Pull-Request: https://github.com/git/git/pull/1301

Range-diff vs v2:

 1:  3b8fb63cc78 ! 1:  01c63ea5fee hide-refs: add hook to force hide refs
     @@ Commit message
          `hide-refs` to hide the private data.
      
          This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
     -    during the reference discovery phase, each reference and will be filtered
     +    during the reference discovery phase, each reference will be filtered
          with this hook. The hook executes once with no arguments for each
          'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
          a version number and server process name ('uploadpack' or 'receive') will
          send to it in pkt-line format, followed by a flush-pkt. The hook should
     -    response with its version number.
     +    respond with its version number.
      
          During reference discovery phase, each reference will be filtered by this
          hook. In the following example, the letter 'G' stands for 'git-receive-pack'
          or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
          decides if the reference will be hidden or not, it sends result back in
     -    pkt-line format protocol, a response "hide" the references will hide
     +    pkt-line format protocol, a response "hide" means the references will hide
          to the client and can not fetch its private data even in protocol V2.
      
                  # Version negotiation
     @@ Commit message
      
          the `hide-refs` will be called during reference discovery phase and
          check each matched reference, a 'hide' response means the reference will
     -    be hidden for its private data and even the `allowTipSHA1InWant` or
     -    `allowReachableSHA1InWant` is set to true.
     +    be hidden for its private data even if `allowTipSHA1InWant` or
     +    `allowReachableSHA1InWant` are set to true.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
 2:  72333c12c3f = 2:  b8a490cb3df t1419: add test cases for hide-refs hook
 3:  e737997eb31 ! 3:  99755b377f0 doc: add documentation for the hide-refs hook
     @@ Metadata
       ## Commit message ##
          doc: add documentation for the hide-refs hook
      
     -    "git upload-pack" or "git recevie-pack" can use "hide-refs"
     +    "git upload-pack" or "git receive-pack" can use "hide-refs"
          hook to filter the references during reference discovery phase.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>

-- 
gitgitgadget

[PATCH v3 1/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook
`hide-refs` to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client and can not fetch its private data even in protocol V2.

        # Version negotiation
        G: PKT-LINE(version=1\0uploadpack)
        G: flush-pkt
        H: PKT-LINE(version=1)
        H: flush-pkt

        # Send reference filter request to hook
        G: PKT-LINE(ref <refname>:<refname_full>)
        G: flush-pkt

        # Receive result from the hook.
        # Case 1: this reference is hidden
        H: PKT-LINE(hide)
        H: flush-pkt

        # Case 2: this reference can be advertised
        H: flush-pkt

To enable the `hide-refs` hook, we should config hiderefs with `force:`
option, eg:

        git config --add transfer.hiderefs force:refs/prefix1/
        git config --add uploadpack.hiderefs force:!refs/prefix2/

the `hide-refs` will be called during reference discovery phase and
check each matched reference, a 'hide' response means the reference will
be hidden for its private data even if `allowTipSHA1InWant` or
`allowReachableSHA1InWant` are set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                  |   1 +
 builtin/receive-pack.c    |   5 +-
 ls-refs.c                 |   2 +-
 refs.c                    | 221 ++++++++++++++++++++++++++++++++++++--
 refs.h                    |   6 ++
 serve.c                   |   2 +
 t/helper/test-hide-refs.c | 152 ++++++++++++++++++++++++++
 t/helper/test-tool.c      |   1 +
 t/helper/test-tool.h      |   1 +
 upload-pack.c             |  32 +++---
 upload-pack.h             |   1 +
 11 files changed, 394 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c

diff --git a/Makefile b/Makefile
index 2ec9b2dc6bb..0c1865370cd 100644
--- a/Makefile
+++ b/Makefile
@@ -793,6 +793,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-hide-refs.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31b48e728be..16f2a21e97a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -296,7 +296,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,
 	struct oidset *seen = data;
 	const char *path = strip_namespace(path_full);
 
-	if (ref_is_hidden(path, path_full))
+	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
 		return 0;
 
 	/*
@@ -1794,7 +1794,8 @@ static void reject_updates_to_hidden(struct command *commands)
 		strbuf_setlen(&refname_full, prefix_len);
 		strbuf_addstr(&refname_full, cmd->ref_name);
 
-		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
+		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
+			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
 			continue;
 		if (is_null_oid(&cmd->new_oid))
 			cmd->error_string = "deny deleting a hidden ref";
diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..b5cb1316d38 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 	strbuf_reset(&data->buf);
 
-	if (ref_is_hidden(refname_nons, refname))
+	if (mark_our_ref(refname_nons, refname, oid))
 		return 0;
 
 	if (!ref_match(&data->prefixes, refname_nons))
diff --git a/refs.c b/refs.c
index 90bcb271687..4a6abbfe4fd 100644
--- a/refs.c
+++ b/refs.c
@@ -8,6 +8,7 @@
 #include "lockfile.h"
 #include "iterator.h"
 #include "refs.h"
+#include "pkt-line.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
 #include "hook.h"
@@ -1296,39 +1297,191 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 }
 
 static struct string_list *hide_refs;
-
+static struct string_list *force_hide_refs;
+static struct strbuf hide_refs_section = STRBUF_INIT;
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
 	const char *key;
+	int force = 0;
+
 	if (!strcmp("transfer.hiderefs", var) ||
 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
 	     !strcmp(key, "hiderefs"))) {
 		char *ref;
 		int len;
+		int forcelen;
 
 		if (!value)
 			return config_error_nonbool(var);
+
+		forcelen = strlen("force:");
+		len = strlen(value);
+		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
+			if (len == forcelen)
+				return error(_("missing value for '%s' with force option"), var);
+
+			force = 1;
+			value += forcelen;
+		}
+
 		ref = xstrdup(value);
 		len = strlen(ref);
 		while (len && ref[len - 1] == '/')
 			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
+
+		if (force) {
+			if (!force_hide_refs) {
+				CALLOC_ARRAY(force_hide_refs, 1);
+				force_hide_refs->strdup_strings = 1;
+			}
+			string_list_append(force_hide_refs, ref);
+		} else {
+			if (!hide_refs) {
+				CALLOC_ARRAY(hide_refs, 1);
+				hide_refs->strdup_strings = 1;
+			}
+			string_list_append(hide_refs, ref);
 		}
-		string_list_append(hide_refs, ref);
 	}
+
+	if (hide_refs_section.len == 0) {
+		strbuf_addstr(&hide_refs_section, section);
+	}
+
 	return 0;
 }
 
-int ref_is_hidden(const char *refname, const char *refname_full)
+static struct child_process *hide_refs_proc;
+static struct packet_reader *hide_refs_reader;
+static void create_hide_refs_process(void) {
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int version = 0;
+	int code;
+
+	hook_path = find_hook("hide-refs");
+	if (!hook_path) {
+		die("can not find hide-refs hook");
+	}
+
+	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
+	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "hide-refs";
+	proc->err = 0;
+
+	code = start_command(proc);
+	if (code)
+		die("can not run hook hide-refs");
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE |
+			   PACKET_READ_GENTLE_ON_EOF);
+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
+	if (!code)
+		code = packet_flush_gently(proc->in);
+
+	if (!code)
+		for (;;) {
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether hide-refs exited abnormally */
+				if (status == PACKET_READ_EOF)
+					die("can not read version message from hook hide-refs");
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+			}
+		}
+
+	if (code)
+		die("can not read version message from hook hide-refs");
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		die(_("hook hide-refs version '%d' is not supported"), version);
+	}
+
+	sigchain_pop(SIGPIPE);
+
+	hide_refs_proc = proc;
+	hide_refs_reader = reader;
+	return;
+}
+
+static int ref_force_hidden_check(const char *refname, const char *refname_full)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int code;
+	int ret = 0;
+
+	if (!force_hide_refs) {
+		return 0;
+	}
+
+	if (!hide_refs_proc) {
+		create_hide_refs_process();
+	}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	code = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	code = packet_flush_gently(hide_refs_proc->in);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(hide_refs_reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether hide-refs exited abnormally */
+			if (status == PACKET_READ_EOF)
+				die("hook hide-refs died abnormally");
+			break;
+		}
+
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, hide_refs_reader->line);
+	}
+
+	if (!strncmp("hide", buf.buf, 4))
+		ret = 1;
+
+	sigchain_pop(SIGPIPE);
+	return ret;
+}
+
+static int ref_hidden_check(const char *refname, const char *refname_full, int force)
 {
+	struct string_list *hide_refs_list = hide_refs;
 	int i;
 
-	if (!hide_refs)
+	if (force)
+		hide_refs_list = force_hide_refs;
+
+	if (!hide_refs_list)
 		return 0;
-	for (i = hide_refs->nr - 1; i >= 0; i--) {
-		const char *match = hide_refs->items[i].string;
+	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
+		const char *match = hide_refs_list->items[i].string;
 		const char *subject;
 		int neg = 0;
 		const char *p;
@@ -1348,12 +1501,58 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 		/* refname can be NULL when namespaces are used. */
 		if (subject &&
 		    skip_prefix(subject, match, &p) &&
-		    (!*p || *p == '/'))
-			return !neg;
+		    (!*p || *p == '/')) {
+			if (neg)
+				return 0;
+			if (!force)
+				return 1;
+			return ref_force_hidden_check(refname, refname_full);
+		}
 	}
 	return 0;
 }
 
+int ref_is_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 0);
+}
+
+int ref_is_force_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 1);
+}
+
+#define OUR_REF		(1u << 12)
+#define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
+static int has_force_hidden;
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid)
+{
+	struct object *o;
+
+	if (!oid || is_null_oid(oid)) {
+		return 0;
+	}
+
+	o = lookup_unknown_object(the_repository, oid);
+	if (ref_is_force_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF_FORCE;
+		has_force_hidden = 1;
+		return 1;
+	}
+	if (ref_is_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF;
+		return 1;
+	}
+	o->flags |= OUR_REF;
+	return 0;
+}
+
+int has_force_hidden_refs(void) {
+	return has_force_hidden;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
diff --git a/refs.h b/refs.h
index 47cb9edbaa8..ae584a644fa 100644
--- a/refs.h
+++ b/refs.h
@@ -818,6 +818,12 @@ int parse_hide_refs_config(const char *var, const char *value, const char *);
  * parameter always points to the full ref name.
  */
 int ref_is_hidden(const char *, const char *);
+int ref_is_force_hidden(const char *, const char *);
+/* return non-zero if the ref is hidden, otherwise 0 */
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid);
+int has_force_hidden_refs(void);
+void lazy_load_hidden_refs(void);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/serve.c b/serve.c
index 733347f602a..6b556719d9f 100644
--- a/serve.c
+++ b/serve.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "refs.h"
 #include "pkt-line.h"
 #include "version.h"
 #include "ls-refs.h"
@@ -320,6 +321,7 @@ void protocol_v2_serve_loop(int stateless_rpc)
 	 * a single request/response exchange
 	 */
 	if (stateless_rpc) {
+		lazy_load_hidden_refs();
 		process_request();
 	} else {
 		for (;;)
diff --git a/t/helper/test-hide-refs.c b/t/helper/test-hide-refs.c
new file mode 100644
index 00000000000..8dcd6700782
--- /dev/null
+++ b/t/helper/test-hide-refs.c
@@ -0,0 +1,152 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *hide_refs_usage[] = {
+	"test-tool hide-refs [<options>...]",
+	NULL
+};
+
+static int die_read_version;
+static int die_write_version;
+static int die_read_first_ref;
+static int die_read_second_ref;
+static int die_after_proc_ref;
+static int verbose;
+static int version = 1;
+static int first_ref;
+static int second_ref;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list returns = STRING_LIST_INIT_NODUP;
+
+struct command {
+	struct command *next;
+	const char *error_string;
+	unsigned int skip_update:1,
+		     did_not_exist:1;
+	int index;
+	struct object_id old_oid;
+	struct object_id new_oid;
+	char ref_name[FLEX_ARRAY]; /* more */
+};
+
+static void hide_refs_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	if (die_read_version)
+		die("die with the --die-read-version option");
+
+	for (;;) {
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		/* Ignore version negotiation for version 0 */
+		if (version == 0)
+			continue;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+		}
+	}
+
+	if (die_write_version)
+		die("die with the --die-write-version option");
+
+	packet_write_fmt(1, "version=%d\n", version);
+	packet_flush(1);
+}
+
+static void hide_refs_proc(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+
+	if (!first_ref) {
+		if (die_read_first_ref)
+			die("die with the --die-read-first-ref option");
+
+		first_ref = 1;
+	}
+
+	if (first_ref && !second_ref) {
+		if (die_read_second_ref)
+			die("die with the --die-read-second-ref option");
+
+		second_ref = 1;
+	}
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = strchr(buf.buf, ':');
+	if (unsorted_string_list_has_string(&returns, p + 1)) {
+		packet_write_fmt(1, "hide");
+	}
+
+	if (die_after_proc_ref)
+		die("die with the --die-after-proc-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__hide_refs(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "die-read-version", &die_read_version,
+			 "die when reading version"),
+		OPT_BOOL(0, "die-write-version", &die_write_version,
+			 "die when writing version"),
+		OPT_BOOL(0, "die-read-first-ref", &die_read_first_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
+			 "die when reading second reference"),
+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
+			 "die after proc ref"),
+		OPT_STRING_LIST('r', "reserved", &returns, "refs-to-force-hidden",
+				"refs that will force hide"),
+		OPT__VERBOSE(&verbose, "be verbose"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	hide_refs_verison(&reader);
+	for (;;) {
+		hide_refs_proc(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 318fdbab0c3..0c30be9fe44 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -69,6 +69,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "hide-refs", cmd__hide_refs },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index bb799271631..79e9068cbeb 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -58,6 +58,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__hide_refs(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/upload-pack.c b/upload-pack.c
index b217a1f469e..095d6c4cb40 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -38,9 +38,10 @@
 #define NOT_SHALLOW	(1u << 17)
 #define CLIENT_SHALLOW	(1u << 18)
 #define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
 
-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
+		NOT_SHALLOW | CLIENT_SHALLOW)
 
 /* Enum for allowed unadvertised object request (UOR) */
 enum allow_uor {
@@ -1155,20 +1156,6 @@ static void receive_needs(struct upload_pack_data *data,
 		packet_flush(1);
 }
 
-/* return non-zero if the ref is hidden, otherwise 0 */
-static int mark_our_ref(const char *refname, const char *refname_full,
-			const struct object_id *oid)
-{
-	struct object *o = lookup_unknown_object(the_repository, oid);
-
-	if (ref_is_hidden(refname, refname_full)) {
-		o->flags |= HIDDEN_REF;
-		return 1;
-	}
-	o->flags |= OUR_REF;
-	return 0;
-}
-
 static int check_ref(const char *refname_full, const struct object_id *oid,
 		     int flag, void *cb_data)
 {
@@ -1454,6 +1441,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
 
 		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
 		if (ref_is_hidden(refname_nons, refname.buf) ||
+			ref_is_force_hidden(refname_nons, refname.buf) ||
 		    read_ref(refname.buf, &oid)) {
 			packet_writer_error(writer, "unknown ref %s", refname_nons);
 			die("unknown ref %s", refname_nons);
@@ -1695,6 +1683,12 @@ enum fetch_state {
 	FETCH_DONE,
 };
 
+static int lazy_load_hidden = 0;
+// lazy load hidden refs for protocol V2
+void lazy_load_hidden_refs(void) {
+	lazy_load_hidden = 1;
+}
+
 int upload_pack_v2(struct repository *r, struct packet_reader *request)
 {
 	enum fetch_state state = FETCH_PROCESS_ARGS;
@@ -1740,6 +1734,12 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 				state = FETCH_DONE;
 			break;
 		case FETCH_SEND_PACK:
+			if (lazy_load_hidden) {
+				head_ref_namespaced(check_ref, NULL);
+				for_each_namespaced_ref(check_ref, NULL);
+			}
+			if (has_force_hidden_refs())
+				check_non_tip(&data);
 			send_wanted_ref_info(&data);
 			send_shallow_info(&data);
 
diff --git a/upload-pack.h b/upload-pack.h
index d6ee25ea98e..541610ba800 100644
--- a/upload-pack.h
+++ b/upload-pack.h
@@ -12,4 +12,5 @@ struct strbuf;
 int upload_pack_advertise(struct repository *r,
 			  struct strbuf *value);
 
+void lazy_load_hidden_refs(void);
 #endif /* UPLOAD_PACK_H */
-- 
gitgitgadget

[PATCH v3 2/3] t1419: add test cases for hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'hide-refs' hook which is used to
filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 ++++++++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 +++++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 +++++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 ++++++++++
 6 files changed, 669 insertions(+)
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh

diff --git a/t/t1419-hide-refs-hook.sh b/t/t1419-hide-refs-hook.sh
new file mode 100755
index 00000000000..52ce61c0e1f
--- /dev/null
+++ b/t/t1419-hide-refs-hook.sh
@@ -0,0 +1,142 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test hide-refs hook'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+setup_bare_repo_and_work_repo () {
+	# Refs of bare_repo : main(A)
+	# Refs of work_repo: main(A)  tags/v123
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git bare_repo.git.dump &&
+		rm -rf work_repo work_repo.dump &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git remote add origin ../bare_repo.git &&
+			git update-ref refs/heads/dev $A &&
+			git update-ref refs/heads/main $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push origin +refs/heads/*:refs/heads/* &&
+			git push origin +refs/tags/*:refs/tags/* &&
+			git push origin +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# setup pre-receive hook
+		write_script bare_repo.git/hooks/pre-receive <<-\EOF &&
+		exec >&2
+		echo "# pre-receive hook"
+		while read old new ref
+		do
+			echo "pre-receive< $old $new $ref"
+		done
+		EOF
+
+		# setup update hook
+		write_script bare_repo.git/hooks/update <<-\EOF &&
+		exec >&2
+		echo "# update hook"
+		echo "update< $@"
+		EOF
+
+		# setup post-receive hook
+		write_script bare_repo.git/hooks/post-receive <<-\EOF
+		exec >&2
+		echo "# post-receive hook"
+		while read old new ref
+		do
+			echo "post-receive< $old $new $ref"
+		done
+		EOF
+	'
+}
+
+run_hide_refs_hook_tests() {
+	case $1 in
+		http)
+			PROTOCOL="HTTP protocol"
+			BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			BAREREPO_PREFIX="$HTTPD_URL"/smart
+			;;
+		local)
+			PROTOCOL="builtin protocol"
+			BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+			BAREREPO_PREFIX="$(pwd)"
+			;;
+	esac
+
+	BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+	GIT_TEST_PROTOCOL_VERSION=$2
+
+	# Run test cases for 'hide-refs' hook
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		# Initialize the bare_repo repository and work_repo
+		setup_bare_repo_and_work_repo
+		git -C work_repo remote set-url origin "$BAREREPO_URL"
+		cp -rf work_repo work_repo.dump
+
+		git -C bare_repo.git config --local http.receivepack true
+		git -C bare_repo.git config --add transfer.hiderefs force:HEAD
+		git -C bare_repo.git config --add transfer.hiderefs force:refs
+		cp -rf bare_repo.git bare_repo.git.dump
+
+		if test "$1" = "http"; then
+			setup_askpass_helper
+			rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			mv bare_repo.git "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+		fi
+
+		. "$t"
+	done
+}
+
+
+setup_bare_repo_and_work_repo
+BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+BAREREPO_PREFIX="$(pwd)"
+BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+# Load test cases that only need to be executed once.
+for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
+do
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:HEAD
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:refs
+	. "$t"
+done
+
+for protocol in 1 2
+do
+	# Run test cases for 'hide-refs' hook on local file protocol.
+	run_hide_refs_hook_tests local $protocol
+done
+
+ROOT_PATH="$PWD"
+. "$TEST_DIRECTORY"/lib-gpg.sh
+. "$TEST_DIRECTORY"/lib-httpd.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
+
+start_httpd
+set_askpass user@host pass@host
+
+# Run test cases for 'hide-refs' hook on HTTP protocol.
+for protocol in 1 2
+do
+	run_hide_refs_hook_tests http $protocol
+done
+
+test_done
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..e86c3d11317
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,80 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-push, git-show-ref and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.  We also
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$TRASH_DIRECTORY/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
+}
+
+filter_out_hide_refs_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+filter_out_user_friendly_and_stable_output () {
+	make_user_friendly_and_stable_output |
+		sed -n ${1+"$@"}
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/once-0000-abnormal-hide-refs-hook.sh b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
new file mode 100644
index 00000000000..c4ff642264e
--- /dev/null
+++ b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
@@ -0,0 +1,161 @@
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs \
+			--die-read-version \
+			-r refs/heads/main
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..1766d869d58
--- /dev/null
+++ b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
@@ -0,0 +1,77 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide branches" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide pull refs and tags" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..d2fc5f0c4f4
--- /dev/null
+++ b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
@@ -0,0 +1,122 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide no refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide all refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	test_must_fail git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which some branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $A
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide pull refs and tags" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $C
+'
diff --git a/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..198fbf5fa5c
--- /dev/null
+++ b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
@@ -0,0 +1,87 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide no refs" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide all refs" '
+	create_commits_in work_repo E &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH v3 3/3] doc: add documentation for the hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <16657101987@163.com>

"git upload-pack" or "git receive-pack" can use "hide-refs"
hook to filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..df712903464 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[hide-refs]]
+hide-refs
+~~~~~~~~~
+
+This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
+during the reference discovery phase, each reference and will be filtered
+by this hook. The hook executes once with no arguments for each
+'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
+a version number and server process name ('uploadpack' or 'receive') will
+send to it in pkt-line format, followed by a flush-pkt. The hook should
+response with its version number.
+
+During reference discovery phase, each reference will be filtered by this
+hook. In the following example, the letter 'G' stands for 'git-receive-pack'
+or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
+decides if the reference will be hidden or not, it sends result back in
+pkt-line format protocol, a response "hide" the references will hide
+to the client and can not fetch it even in protocol V2.
+
+	# Version negotiation
+	G: PKT-LINE(version=1\0uploadpack)
+	G: flush-pkt
+	H: PKT-LINE(version=1)
+	H: flush-pkt
+
+	# Send reference filter request to hook
+	G: PKT-LINE(ref <refname>:<refnamefull>)
+	G: flush-pkt
+
+	# Receive result from the hook.
+	# Case 1: this reference is hidden
+	H: PKT-LINE(hide)
+	H: flush-pkt
+
+	# Case 2: this reference can be advertised
+	H: flush-pkt
+
+To enable the `hide-refs` hook, we should config hiderefs with `force:`
+option, eg:
+
+	git config --add transfer.hiderefs force:refs/prefix1/
+	git config --add uploadpack.hiderefs force:!refs/prefix2/
+
+the `hide-refs` will be called during reference discovery phase and
+check each matched reference, a 'hide' reponse means the reference will
+be hidden for its private data and even the `allowTipSHA1InWant` and
+`allowReachableSHA1InWant` is set to true.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget

[PATCH v4 0/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook hide-refs
to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered with
this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client and can not fetch its private data even in protocol V2.

            # Version negotiation
            G: PKT-LINE(version=1\0uploadpack)
            G: flush-pkt
            H: PKT-LINE(version=1)
            H: flush-pkt

            # Send reference filter request to hook
            G: PKT-LINE(ref <refname>:<refname_full>)
            G: flush-pkt

            # Receive result from the hook.
            # Case 1: this reference is hidden
            H: PKT-LINE(hide)
            H: flush-pkt

            # Case 2: this reference can be advertised
            H: flush-pkt


To enable the hide-refs hook, we should config hiderefs with force: option,
eg:

            git config --add transfer.hiderefs force:refs/prefix1/
            git config --add uploadpack.hiderefs force:!refs/prefix2/


the hide-refs will be called during reference discovery phase and check each
matched reference, a 'hide' response means the reference will be hidden for
its private data even if allowTipSHA1InWant or allowReachableSHA1InWant are
set to true.

Sun Chao (3):
  hide-refs: add hook to force hide refs
  t1419: add test cases for hide-refs hook
  doc: add documentation for the hide-refs hook

 Documentation/githooks.txt                    |  48 ++++
 Makefile                                      |   1 +
 builtin/receive-pack.c                        |   5 +-
 ls-refs.c                                     |   2 +-
 refs.c                                        | 221 +++++++++++++++++-
 refs.h                                        |   6 +
 serve.c                                       |   2 +
 t/helper/test-hide-refs.c                     | 152 ++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 +++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 ++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 ++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 +++++++
 upload-pack.c                                 |  32 +--
 upload-pack.h                                 |   1 +
 18 files changed, 1111 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh


base-commit: afa70145a25e81faa685dc0b465e52b45d2444bd
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v4
Pull-Request: https://github.com/git/git/pull/1301

Range-diff vs v3:

 1:  01c63ea5fee = 1:  01c63ea5fee hide-refs: add hook to force hide refs
 2:  b8a490cb3df = 2:  b8a490cb3df t1419: add test cases for hide-refs hook
 3:  99755b377f0 ! 3:  8c5ae78de47 doc: add documentation for the hide-refs hook
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
      +a version number and server process name ('uploadpack' or 'receive') will
      +send to it in pkt-line format, followed by a flush-pkt. The hook should
     -+response with its version number.
     ++respond with its version number.
      +
      +During reference discovery phase, each reference will be filtered by this
      +hook. In the following example, the letter 'G' stands for 'git-receive-pack'
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +	git config --add uploadpack.hiderefs force:!refs/prefix2/
      +
      +the `hide-refs` will be called during reference discovery phase and
     -+check each matched reference, a 'hide' reponse means the reference will
     -+be hidden for its private data and even the `allowTipSHA1InWant` and
     -+`allowReachableSHA1InWant` is set to true.
     ++check each matched reference, a 'hide' response means the reference will
     ++be hidden for its private data even if `allowTipSHA1InWant` and
     ++`allowReachableSHA1InWant` are set to true.
      +
       [[pre-receive]]
       pre-receive

-- 
gitgitgadget

[PATCH v4 1/3] hide-refs: add hook to force hide refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook
`hide-refs` to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client and can not fetch its private data even in protocol V2.

        # Version negotiation
        G: PKT-LINE(version=1\0uploadpack)
        G: flush-pkt
        H: PKT-LINE(version=1)
        H: flush-pkt

        # Send reference filter request to hook
        G: PKT-LINE(ref <refname>:<refname_full>)
        G: flush-pkt

        # Receive result from the hook.
        # Case 1: this reference is hidden
        H: PKT-LINE(hide)
        H: flush-pkt

        # Case 2: this reference can be advertised
        H: flush-pkt

To enable the `hide-refs` hook, we should config hiderefs with `force:`
option, eg:

        git config --add transfer.hiderefs force:refs/prefix1/
        git config --add uploadpack.hiderefs force:!refs/prefix2/

the `hide-refs` will be called during reference discovery phase and
check each matched reference, a 'hide' response means the reference will
be hidden for its private data even if `allowTipSHA1InWant` or
`allowReachableSHA1InWant` are set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                  |   1 +
 builtin/receive-pack.c    |   5 +-
 ls-refs.c                 |   2 +-
 refs.c                    | 221 ++++++++++++++++++++++++++++++++++++--
 refs.h                    |   6 ++
 serve.c                   |   2 +
 t/helper/test-hide-refs.c | 152 ++++++++++++++++++++++++++
 t/helper/test-tool.c      |   1 +
 t/helper/test-tool.h      |   1 +
 upload-pack.c             |  32 +++---
 upload-pack.h             |   1 +
 11 files changed, 394 insertions(+), 30 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c

diff --git a/Makefile b/Makefile
index 2ec9b2dc6bb..0c1865370cd 100644
--- a/Makefile
+++ b/Makefile
@@ -793,6 +793,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-hide-refs.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 31b48e728be..16f2a21e97a 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -296,7 +296,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,
 	struct oidset *seen = data;
 	const char *path = strip_namespace(path_full);
 
-	if (ref_is_hidden(path, path_full))
+	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
 		return 0;
 
 	/*
@@ -1794,7 +1794,8 @@ static void reject_updates_to_hidden(struct command *commands)
 		strbuf_setlen(&refname_full, prefix_len);
 		strbuf_addstr(&refname_full, cmd->ref_name);
 
-		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
+		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
+			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
 			continue;
 		if (is_null_oid(&cmd->new_oid))
 			cmd->error_string = "deny deleting a hidden ref";
diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..b5cb1316d38 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 	strbuf_reset(&data->buf);
 
-	if (ref_is_hidden(refname_nons, refname))
+	if (mark_our_ref(refname_nons, refname, oid))
 		return 0;
 
 	if (!ref_match(&data->prefixes, refname_nons))
diff --git a/refs.c b/refs.c
index 90bcb271687..4a6abbfe4fd 100644
--- a/refs.c
+++ b/refs.c
@@ -8,6 +8,7 @@
 #include "lockfile.h"
 #include "iterator.h"
 #include "refs.h"
+#include "pkt-line.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
 #include "hook.h"
@@ -1296,39 +1297,191 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 }
 
 static struct string_list *hide_refs;
-
+static struct string_list *force_hide_refs;
+static struct strbuf hide_refs_section = STRBUF_INIT;
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
 	const char *key;
+	int force = 0;
+
 	if (!strcmp("transfer.hiderefs", var) ||
 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
 	     !strcmp(key, "hiderefs"))) {
 		char *ref;
 		int len;
+		int forcelen;
 
 		if (!value)
 			return config_error_nonbool(var);
+
+		forcelen = strlen("force:");
+		len = strlen(value);
+		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
+			if (len == forcelen)
+				return error(_("missing value for '%s' with force option"), var);
+
+			force = 1;
+			value += forcelen;
+		}
+
 		ref = xstrdup(value);
 		len = strlen(ref);
 		while (len && ref[len - 1] == '/')
 			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
+
+		if (force) {
+			if (!force_hide_refs) {
+				CALLOC_ARRAY(force_hide_refs, 1);
+				force_hide_refs->strdup_strings = 1;
+			}
+			string_list_append(force_hide_refs, ref);
+		} else {
+			if (!hide_refs) {
+				CALLOC_ARRAY(hide_refs, 1);
+				hide_refs->strdup_strings = 1;
+			}
+			string_list_append(hide_refs, ref);
 		}
-		string_list_append(hide_refs, ref);
 	}
+
+	if (hide_refs_section.len == 0) {
+		strbuf_addstr(&hide_refs_section, section);
+	}
+
 	return 0;
 }
 
-int ref_is_hidden(const char *refname, const char *refname_full)
+static struct child_process *hide_refs_proc;
+static struct packet_reader *hide_refs_reader;
+static void create_hide_refs_process(void) {
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int version = 0;
+	int code;
+
+	hook_path = find_hook("hide-refs");
+	if (!hook_path) {
+		die("can not find hide-refs hook");
+	}
+
+	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
+	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "hide-refs";
+	proc->err = 0;
+
+	code = start_command(proc);
+	if (code)
+		die("can not run hook hide-refs");
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE |
+			   PACKET_READ_GENTLE_ON_EOF);
+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
+	if (!code)
+		code = packet_flush_gently(proc->in);
+
+	if (!code)
+		for (;;) {
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether hide-refs exited abnormally */
+				if (status == PACKET_READ_EOF)
+					die("can not read version message from hook hide-refs");
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+			}
+		}
+
+	if (code)
+		die("can not read version message from hook hide-refs");
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		die(_("hook hide-refs version '%d' is not supported"), version);
+	}
+
+	sigchain_pop(SIGPIPE);
+
+	hide_refs_proc = proc;
+	hide_refs_reader = reader;
+	return;
+}
+
+static int ref_force_hidden_check(const char *refname, const char *refname_full)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int code;
+	int ret = 0;
+
+	if (!force_hide_refs) {
+		return 0;
+	}
+
+	if (!hide_refs_proc) {
+		create_hide_refs_process();
+	}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	code = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	code = packet_flush_gently(hide_refs_proc->in);
+	if (code)
+		die("hook hide-refs died abnormally");
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(hide_refs_reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether hide-refs exited abnormally */
+			if (status == PACKET_READ_EOF)
+				die("hook hide-refs died abnormally");
+			break;
+		}
+
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, hide_refs_reader->line);
+	}
+
+	if (!strncmp("hide", buf.buf, 4))
+		ret = 1;
+
+	sigchain_pop(SIGPIPE);
+	return ret;
+}
+
+static int ref_hidden_check(const char *refname, const char *refname_full, int force)
 {
+	struct string_list *hide_refs_list = hide_refs;
 	int i;
 
-	if (!hide_refs)
+	if (force)
+		hide_refs_list = force_hide_refs;
+
+	if (!hide_refs_list)
 		return 0;
-	for (i = hide_refs->nr - 1; i >= 0; i--) {
-		const char *match = hide_refs->items[i].string;
+	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
+		const char *match = hide_refs_list->items[i].string;
 		const char *subject;
 		int neg = 0;
 		const char *p;
@@ -1348,12 +1501,58 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 		/* refname can be NULL when namespaces are used. */
 		if (subject &&
 		    skip_prefix(subject, match, &p) &&
-		    (!*p || *p == '/'))
-			return !neg;
+		    (!*p || *p == '/')) {
+			if (neg)
+				return 0;
+			if (!force)
+				return 1;
+			return ref_force_hidden_check(refname, refname_full);
+		}
 	}
 	return 0;
 }
 
+int ref_is_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 0);
+}
+
+int ref_is_force_hidden(const char *refname, const char *refname_full)
+{
+	return ref_hidden_check(refname, refname_full, 1);
+}
+
+#define OUR_REF		(1u << 12)
+#define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
+static int has_force_hidden;
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid)
+{
+	struct object *o;
+
+	if (!oid || is_null_oid(oid)) {
+		return 0;
+	}
+
+	o = lookup_unknown_object(the_repository, oid);
+	if (ref_is_force_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF_FORCE;
+		has_force_hidden = 1;
+		return 1;
+	}
+	if (ref_is_hidden(refname, refname_full)) {
+		o->flags |= HIDDEN_REF;
+		return 1;
+	}
+	o->flags |= OUR_REF;
+	return 0;
+}
+
+int has_force_hidden_refs(void) {
+	return has_force_hidden;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
diff --git a/refs.h b/refs.h
index 47cb9edbaa8..ae584a644fa 100644
--- a/refs.h
+++ b/refs.h
@@ -818,6 +818,12 @@ int parse_hide_refs_config(const char *var, const char *value, const char *);
  * parameter always points to the full ref name.
  */
 int ref_is_hidden(const char *, const char *);
+int ref_is_force_hidden(const char *, const char *);
+/* return non-zero if the ref is hidden, otherwise 0 */
+int mark_our_ref(const char *refname, const char *refname_full,
+			const struct object_id *oid);
+int has_force_hidden_refs(void);
+void lazy_load_hidden_refs(void);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/serve.c b/serve.c
index 733347f602a..6b556719d9f 100644
--- a/serve.c
+++ b/serve.c
@@ -1,6 +1,7 @@
 #include "cache.h"
 #include "repository.h"
 #include "config.h"
+#include "refs.h"
 #include "pkt-line.h"
 #include "version.h"
 #include "ls-refs.h"
@@ -320,6 +321,7 @@ void protocol_v2_serve_loop(int stateless_rpc)
 	 * a single request/response exchange
 	 */
 	if (stateless_rpc) {
+		lazy_load_hidden_refs();
 		process_request();
 	} else {
 		for (;;)
diff --git a/t/helper/test-hide-refs.c b/t/helper/test-hide-refs.c
new file mode 100644
index 00000000000..8dcd6700782
--- /dev/null
+++ b/t/helper/test-hide-refs.c
@@ -0,0 +1,152 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *hide_refs_usage[] = {
+	"test-tool hide-refs [<options>...]",
+	NULL
+};
+
+static int die_read_version;
+static int die_write_version;
+static int die_read_first_ref;
+static int die_read_second_ref;
+static int die_after_proc_ref;
+static int verbose;
+static int version = 1;
+static int first_ref;
+static int second_ref;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list returns = STRING_LIST_INIT_NODUP;
+
+struct command {
+	struct command *next;
+	const char *error_string;
+	unsigned int skip_update:1,
+		     did_not_exist:1;
+	int index;
+	struct object_id old_oid;
+	struct object_id new_oid;
+	char ref_name[FLEX_ARRAY]; /* more */
+};
+
+static void hide_refs_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	if (die_read_version)
+		die("die with the --die-read-version option");
+
+	for (;;) {
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		/* Ignore version negotiation for version 0 */
+		if (version == 0)
+			continue;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+		}
+	}
+
+	if (die_write_version)
+		die("die with the --die-write-version option");
+
+	packet_write_fmt(1, "version=%d\n", version);
+	packet_flush(1);
+}
+
+static void hide_refs_proc(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+
+	if (!first_ref) {
+		if (die_read_first_ref)
+			die("die with the --die-read-first-ref option");
+
+		first_ref = 1;
+	}
+
+	if (first_ref && !second_ref) {
+		if (die_read_second_ref)
+			die("die with the --die-read-second-ref option");
+
+		second_ref = 1;
+	}
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = strchr(buf.buf, ':');
+	if (unsorted_string_list_has_string(&returns, p + 1)) {
+		packet_write_fmt(1, "hide");
+	}
+
+	if (die_after_proc_ref)
+		die("die with the --die-after-proc-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__hide_refs(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "die-read-version", &die_read_version,
+			 "die when reading version"),
+		OPT_BOOL(0, "die-write-version", &die_write_version,
+			 "die when writing version"),
+		OPT_BOOL(0, "die-read-first-ref", &die_read_first_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
+			 "die when reading second reference"),
+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
+			 "die after proc ref"),
+		OPT_STRING_LIST('r', "reserved", &returns, "refs-to-force-hidden",
+				"refs that will force hide"),
+		OPT__VERBOSE(&verbose, "be verbose"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	hide_refs_verison(&reader);
+	for (;;) {
+		hide_refs_proc(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 318fdbab0c3..0c30be9fe44 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -69,6 +69,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "hide-refs", cmd__hide_refs },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index bb799271631..79e9068cbeb 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -58,6 +58,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__hide_refs(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/upload-pack.c b/upload-pack.c
index b217a1f469e..095d6c4cb40 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -38,9 +38,10 @@
 #define NOT_SHALLOW	(1u << 17)
 #define CLIENT_SHALLOW	(1u << 18)
 #define HIDDEN_REF	(1u << 19)
+#define HIDDEN_REF_FORCE	(1u << 20)
 
-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
+		NOT_SHALLOW | CLIENT_SHALLOW)
 
 /* Enum for allowed unadvertised object request (UOR) */
 enum allow_uor {
@@ -1155,20 +1156,6 @@ static void receive_needs(struct upload_pack_data *data,
 		packet_flush(1);
 }
 
-/* return non-zero if the ref is hidden, otherwise 0 */
-static int mark_our_ref(const char *refname, const char *refname_full,
-			const struct object_id *oid)
-{
-	struct object *o = lookup_unknown_object(the_repository, oid);
-
-	if (ref_is_hidden(refname, refname_full)) {
-		o->flags |= HIDDEN_REF;
-		return 1;
-	}
-	o->flags |= OUR_REF;
-	return 0;
-}
-
 static int check_ref(const char *refname_full, const struct object_id *oid,
 		     int flag, void *cb_data)
 {
@@ -1454,6 +1441,7 @@ static int parse_want_ref(struct packet_writer *writer, const char *line,
 
 		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
 		if (ref_is_hidden(refname_nons, refname.buf) ||
+			ref_is_force_hidden(refname_nons, refname.buf) ||
 		    read_ref(refname.buf, &oid)) {
 			packet_writer_error(writer, "unknown ref %s", refname_nons);
 			die("unknown ref %s", refname_nons);
@@ -1695,6 +1683,12 @@ enum fetch_state {
 	FETCH_DONE,
 };
 
+static int lazy_load_hidden = 0;
+// lazy load hidden refs for protocol V2
+void lazy_load_hidden_refs(void) {
+	lazy_load_hidden = 1;
+}
+
 int upload_pack_v2(struct repository *r, struct packet_reader *request)
 {
 	enum fetch_state state = FETCH_PROCESS_ARGS;
@@ -1740,6 +1734,12 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 				state = FETCH_DONE;
 			break;
 		case FETCH_SEND_PACK:
+			if (lazy_load_hidden) {
+				head_ref_namespaced(check_ref, NULL);
+				for_each_namespaced_ref(check_ref, NULL);
+			}
+			if (has_force_hidden_refs())
+				check_non_tip(&data);
 			send_wanted_ref_info(&data);
 			send_shallow_info(&data);
 
diff --git a/upload-pack.h b/upload-pack.h
index d6ee25ea98e..541610ba800 100644
--- a/upload-pack.h
+++ b/upload-pack.h
@@ -12,4 +12,5 @@ struct strbuf;
 int upload_pack_advertise(struct repository *r,
 			  struct strbuf *value);
 
+void lazy_load_hidden_refs(void);
 #endif /* UPLOAD_PACK_H */
-- 
gitgitgadget

Re: [PATCH v4 1/3] hide-refs: add hook to force hide refs

[Junio C Hamano]

This step seems to do too many things at once and is hard to assess
its correctness, I am afraid.  Anybody who has deep understanding of
the protocol involved share that impression?

> To enable the `hide-refs` hook, we should config hiderefs with `force:`
> option, eg:
>
>         git config --add transfer.hiderefs force:refs/prefix1/
>         git config --add uploadpack.hiderefs force:!refs/prefix2/
>
> the `hide-refs` will be called during reference discovery phase and
> check each matched reference, a 'hide' response means the reference will
> be hidden for its private data even if `allowTipSHA1InWant` or
> `allowReachableSHA1InWant` are set to true.

If the prefix is a sign to let the external process to tell if it is
to be hidden or shown, it does not sound like "force" at all, at
least to me ("force" sounds more like "no matter what other things
may want to show it, these are hidden").

> diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
> index 31b48e728be..16f2a21e97a 100644
> --- a/builtin/receive-pack.c
> +++ b/builtin/receive-pack.c
> @@ -296,7 +296,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,
>  	struct oidset *seen = data;
>  	const char *path = strip_namespace(path_full);
>  
> -	if (ref_is_hidden(path, path_full))
> +	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
>  		return 0;

Are there places where only ref_is_hidden() is called, or do
codepaths that used to care ref_is_hidden() now all have to write
the above (A || B) conditional?  I am wondering why the new
"force-hidden" check is not part of ref_is_hidden() so that the
callers do not have to care.

> @@ -1794,7 +1794,8 @@ static void reject_updates_to_hidden(struct command *commands)
>  		strbuf_setlen(&refname_full, prefix_len);
>  		strbuf_addstr(&refname_full, cmd->ref_name);
>  
> -		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
> +		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
> +			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))

Likewise.

> diff --git a/ls-refs.c b/ls-refs.c
> index 98e69373c84..b5cb1316d38 100644
> --- a/ls-refs.c
> +++ b/ls-refs.c
> @@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
>  
>  	strbuf_reset(&data->buf);
>  
> -	if (ref_is_hidden(refname_nons, refname))
> +	if (mark_our_ref(refname_nons, refname, oid))
>  		return 0;
>  
>  	if (!ref_match(&data->prefixes, refname_nons))

It used to be that send_ref() did not touch the object flag bits.
It just said "if it is hidden, or if it is outside the namespace, do
not show and return" before telling the other side about the ref,
and even the ref we send to the other side, we did not muck with
flag bits with OUR_REF bit (and we didn't touch HIDDEN_REF bit,
either).

Now we do.  How can it be determined if this change is correct and
safe?

If the ref is not hidden (either in the traditional sense, or with
the new "force" sense), we do not return 0.  What if it is outside
the namespace so we returned without sending it to the other side?
The original code didn't touch the flags bit, but now we mark the
object with OUR_REF bit even though we ended up not sending the ref
to the other side.  Is that an intended change?

>  int parse_hide_refs_config(const char *var, const char *value, const char *section)
>  {
>  	const char *key;
> +	int force = 0;
> +
>  	if (!strcmp("transfer.hiderefs", var) ||
>  	    (!parse_config_key(var, section, NULL, NULL, &key) &&
>  	     !strcmp(key, "hiderefs"))) {
>  		char *ref;
>  		int len;
> +		int forcelen;
>  
>  		if (!value)
>  			return config_error_nonbool(var);
> +
> +		forcelen = strlen("force:");
> +		len = strlen(value);
> +		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {

skip_prefix() would probably be a good API function to learn here, perhaps?


> +static struct child_process *hide_refs_proc;
> +static struct packet_reader *hide_refs_reader;
> +static void create_hide_refs_process(void) {

Style.  The braces around a function block occupy their own line by
themselves.

> +	struct child_process *proc;
> +	struct packet_reader *reader;
> +	const char *hook_path;
> +	int version = 0;
> +	int code;
> +
> +	hook_path = find_hook("hide-refs");
> +	if (!hook_path) {
> +		die("can not find hide-refs hook");
> +	}

No need for braces around a single statement block.

Is that a condition worth dying, indicating a misconfiguration by
the user?  Or would it make more sense to treat as if the process
says no refs are hidden (or all refs are hidden)?

I do not think we spell "cannot" as "can not" in our messages.

> +	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
> +	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));

Style.  No SP after xcalloc, or sizeof.

> +	child_process_init(proc);
> +	strvec_push(&proc->args, hook_path);
> +	proc->in = -1;
> +	proc->out = -1;
> +	proc->trace2_hook_name = "hide-refs";
> +	proc->err = 0;
> +
> +	code = start_command(proc);
> +	if (code)
> +		die("can not run hook hide-refs");

Unusually named variable.  I think "code" here is a variable
normally called "status" (or "ret" if it eventually becomes the
return value from this function).  Shouldn't this function return an
error and have it handled by its caller, by the way, instead of
returning void and making liberal calls to die()?

> +	sigchain_push(SIGPIPE, SIG_IGN);
> +
> +	/* Version negotiaton */
> +	packet_reader_init(reader, proc->out, NULL, 0,
> +			   PACKET_READ_CHOMP_NEWLINE |
> +			   PACKET_READ_GENTLE_ON_EOF);
> +	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
> +	if (!code)
> +		code = packet_flush_gently(proc->in);

In general, it is a bad pattern to hide mainline "good case"
processing inside "if (previous steps all went fine)" conditionals,
as it makes the code unnecessarily hard to follow.

Instead, we typically write more like this:

-- >8 -- cut here -- >8 --

	int ret = -1; /* assume failure */

        if (packet_write_fmt_gently(...))
		goto error_exit;

	for (;;) {
		... interact with the other side ...
		if (error)
			goto error_exit;
	}

	... continue with mainline "good case" processing ...

	... after all went well ...
	ret = 0;

error_exit:
	if (ret < 0) {
		... emit error message ...
		... clean-up specific to error case if necessary
	}
	... clean-up as needed ...

	return ret;

-- 8< -- cut here -- 8< --

I am not reviewing the rest of the patch in this sitting---I may
later come back to continue reading it, but I'll stop here and send
out comments on what I have seen first.

Thanks.

Re: [PATCH v4 1/3] hide-refs: add hook to force hide refs

[孙超]

> On Aug 16, 2022, at 00:02, Junio C Hamano <gitster@pobox.com> wrote:
> 
> Not "right now".
> 
> Instead of flooding the list with repeated "oops that was wrong"
> updates, it may be more effective use of others' time to wait for
> more feedback before acting on them, and to take time to proofread
> the result of your updates before sending them out.
> 
> Thanks.

Got it, I'll take more time in reviewing my patch updates in the future.

> On Aug 16, 2022, at 02:18, Junio C Hamano <gitster@pobox.com> wrote:
> 
> If the prefix is a sign to let the external process to tell if it is
> to be hidden or shown, it does not sound like "force" at all, at
> least to me ("force" sounds more like "no matter what other things
> may want to show it, these are hidden").

I’ve read the codes about refs hidden and I had 3 ideas in mind, but I
didn't know which is better: 1) create a new config item (like transfer.forceHideRefsByHook),
2) call `hide-refs` hook directly to check all refs if it exists, 3) add a new prefix option
to `transfer.hiderefs` to call the new hook.

I choose the third one but it is indeed hard to understand and the `force`
is not appropriate.

> Are there places where only ref_is_hidden() is called, or do
> codepaths that used to care ref_is_hidden() now all have to write
> the above (A || B) conditional?  I am wondering why the new
> "force-hidden" check is not part of ref_is_hidden() so that the
> callers do not have to care.
> 
>> @@ -1794,7 +1794,8 @@ static void reject_updates_to_hidden(struct command *commands)
>> 		strbuf_setlen(&refname_full, prefix_len);
>> 		strbuf_addstr(&refname_full, cmd->ref_name);
>> 
>> -		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
>> +		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
>> +			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
> 
> Likewise.

In `mark_our_ref` I call ref_is_force_hidden() and ref_is_hidden() separately for a ref, and put
a new `HIDDEN_REF_FORCE` bit to the object flags if the ref is force hidden, then in `has_unreachable`
function the objects with this flag bit will be considered as 'not ancestors of our ref’. And
I do not want to change the original mechanism of ref_is_hidden(), this is why I add a new object
flag bit `HIDDEN_REF_FORCE`.

> It used to be that send_ref() did not touch the object flag bits.
> It just said "if it is hidden, or if it is outside the namespace, do
> not show and return" before telling the other side about the ref,
> and even the ref we send to the other side, we did not muck with
> flag bits with OUR_REF bit (and we didn't touch HIDDEN_REF bit,
> either).
> 
> Now we do.  How can it be determined if this change is correct and
> safe?

I’m not very confident about the changes either so I have added some test cases for
upload-pack V1 and V2, but I think I need add more test cases for other commands after
considering your comment here.

> 
> If the ref is not hidden (either in the traditional sense, or with
> the new "force" sense), we do not return 0.  What if it is outside
> the namespace so we returned without sending it to the other side?
> The original code didn't touch the flags bit, but now we mark the
> object with OUR_REF bit even though we ended up not sending the ref
> to the other side.  Is that an intended change?

Yes, this is a intended change here.

If I understand correctly, command `ls-refs` is called during protocol V2, I mark the object
flags here and use them when responding to command `fetch`, and I call `check_non_tip()`
before send the `want` data to the client which will check the force hidden objects.

>> +		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
> 
> skip_prefix() would probably be a good API function to learn here, perhaps?
> 

Will fix it.

> 
>> +static struct child_process *hide_refs_proc;
>> +static struct packet_reader *hide_refs_reader;
>> +static void create_hide_refs_process(void) {
> 
> Style.  The braces around a function block occupy their own line by
> themselves.
> 
Will fix it. And I will read the style document again before update the patches.

> 
> No need for braces around a single statement block.
> 
Will fix it.

> Is that a condition worth dying, indicating a misconfiguration by
> the user?  Or would it make more sense to treat as if the process
> says no refs are hidden (or all refs are hidden)?

Thanks for your question here, I will think about it later. Just like `pre-receive` hook,
git server will skip it if it does not exists. I think we should not die here.

> 
> I do not think we spell "cannot" as "can not" in our messages.

Will fix it.

> 
>> +	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
>> +	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
> 
> Style.  No SP after xcalloc, or sizeof.

Will fix it.

>> +	code = start_command(proc);
>> +	if (code)
>> +		die("can not run hook hide-refs");
> 
> Unusually named variable.  I think "code" here is a variable
> normally called "status" (or "ret" if it eventually becomes the
> return value from this function).  Shouldn't this function return an
> error and have it handled by its caller, by the way, instead of
> returning void and making liberal calls to die()?

Will do.

> 
>> +	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
>> +	if (!code)
>> +		code = packet_flush_gently(proc->in);
> 
> In general, it is a bad pattern to hide mainline "good case"
> processing inside "if (previous steps all went fine)" conditionals,
> as it makes the code unnecessarily hard to follow.
> 
> Instead, we typically write more like this:
> 
> -- >8 -- cut here -- >8 --
> 
> 	int ret = -1; /* assume failure */
> 
>        if (packet_write_fmt_gently(...))
> 		goto error_exit;
> 
> 	for (;;) {
> 		... interact with the other side ...
> 		if (error)
> 			goto error_exit;
> 	}
> 
> 	... continue with mainline "good case" processing ...
> 
> 	... after all went well ...
> 	ret = 0;
> 
> error_exit:
> 	if (ret < 0) {
> 		... emit error message ...
> 		... clean-up specific to error case if necessary
> 	}
> 	... clean-up as needed ...
> 
> 	return ret;
> 
> -- 8< -- cut here -- 8< --
> 
> I am not reviewing the rest of the patch in this sitting---I may
> later come back to continue reading it, but I'll stop here and send
> out comments on what I have seen first.
> 
> Thanks.
> 

Thanks a lot, I Will do it.

Re: [PATCH v4 1/3] hide-refs: add hook to force hide refs

[Calvin Wan]

Hi Sun,

A couple of us from the mailing list reviewed your patch yesterday during
review club and I'm going to summarize our thoughts here.

Starting with you commit message, it is not entirely clear what your
series is trying to achieve. While you do attempt to set the scene in
the first paragraph, it would be better to go into more detail of how a
user would use this hook. Do you already have something like this
working downstream for you at your company? If so, that would be a good
reference to provide context for readers. If not, try to sell your use
case better to us by providing examples and anything else this could be
useful for. Your commit message should also have a broad description of
the changes, explain difficult/tricky changes, and dicuss
tradeoffs/complexity.  

As Junio has noted, there is a lot going on here. For example, changes
you make to pre-existing functionality should come with an explanation.
One way to manage this complexity for reviewers is by splitting up your
changes into more logically different commits.

For your tests, they should show a working example of thie feature, the
motivation behind the feature, and a description of the interface. The
structure of the tests is also confusing and there seem to be many
unnecessary tests. It is OK to be verbose and obvious in tests -- it is
very important for reviewers and others looking at your tests to easily
understand what each test is doing.

"Sun Chao via GitGitGadget" <gitgitgadget@gmail.com> writes:
> From: Sun Chao <sunchao9@huawei.com>
> 
> Gerrit is implemented by JGit and is known as a centralized workflow system
> which supports reference-level access control for repository. If we choose
> to work in centralized workflow like what Gerrit provided, reference-level
> access control is needed and we might add a reference filter hook
> `hide-refs` to hide the private data.

Why is Gerrit being centralized relevant to ref-level access control?

> 
> This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
> during the reference discovery phase, each reference will be filtered
> with this hook. The hook executes once with no arguments for each
> 'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
> a version number and server process name ('uploadpack' or 'receive') will
> send to it in pkt-line format, followed by a flush-pkt. The hook should
> respond with its version number.
> 
> During reference discovery phase, each reference will be filtered by this
> hook. In the following example, the letter 'G' stands for 'git-receive-pack'
> or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
> decides if the reference will be hidden or not, it sends result back in
> pkt-line format protocol, a response "hide" means the references will hide
> to the client and can not fetch its private data even in protocol V2.

What is the reasoning behind special casing v2 here? Is it possible
you're confusing remote helper protocol and wire protocol?

> +static int lazy_load_hidden = 0;
> +// lazy load hidden refs for protocol V2
> +void lazy_load_hidden_refs(void) {
> +	lazy_load_hidden = 1;
> +}
> +

What does lazy_load_hidden do?

I know this is a lot to go thru for your first patch series, but please
don't get discouraged! Feel free to ask any questions if you're confused
about any of the feedback. We didn't dive too deeply into the specifics
of your code since we believe there are higher level fundamental issues
you should address first. There has also been similar discussion regarding
differing ACLs within a single repository so it is probably worth a read
here[1].

[1] <CAJoAoZmsuwYCA8XGziEA-qwghg9h22Af98JQE1AuHHBRfQgrDA@mail.gmail.com>

Re: [PATCH v4 1/3] hide-refs: add hook to force hide refs

[孙超]

> On Aug 19, 2022, at 02:51, Calvin Wan <calvinwan@google.com> wrote:
> 
> Hi Sun,
> 
> A couple of us from the mailing list reviewed your patch yesterday during
> review club and I'm going to summarize our thoughts here.
> 
> Starting with you commit message, it is not entirely clear what your
> series is trying to achieve. While you do attempt to set the scene in
> the first paragraph, it would be better to go into more detail of how a
> user would use this hook. Do you already have something like this
> working downstream for you at your company? If so, that would be a good
> reference to provide context for readers. If not, try to sell your use
> case better to us by providing examples and anything else this could be
> useful for. Your commit message should also have a broad description of
> the changes, explain difficult/tricky changes, and dicuss
> tradeoffs/complexity.  

First, I really appreciate that you spent your precious time reviewing my
patches and tell me where is not enough and how to do it. I will check
my patches again and I wish I can update it with better descriptions in
a week (I can do it only in the evening time so it need couple days).

> 
> As Junio has noted, there is a lot going on here. For example, changes
> you make to pre-existing functionality should come with an explanation.
> One way to manage this complexity for reviewers is by splitting up your
> changes into more logically different commits.

Yes, Junio had given me some important comments just like yours and I
still very seriously consider how to solve them. I will split up my changes
and write more explanations.

> 
> For your tests, they should show a working example of thie feature, the
> motivation behind the feature, and a description of the interface. The

Thanks, will do it.

> structure of the tests is also confusing and there seem to be many
> unnecessary tests. It is OK to be verbose and obvious in tests -- it is
> very important for reviewers and others looking at your tests to easily
> understand what each test is doing.

Thanks, I will refactor the tests and add more descriptions to make them easily
understand.

> 
>> `hide-refs` to hide the private data.
> 
> Why is Gerrit being centralized relevant to ref-level access control?

Will explain it in my next new update why I think so.

> 
>> to the client and can not fetch its private data even in protocol V2.
> 
> What is the reasoning behind special casing v2 here? Is it possible
> you're confusing remote helper protocol and wire protocol?

I will try to learn about the differences between them and answer the
question here.

> 
>> +static int lazy_load_hidden = 0;
>> +// lazy load hidden refs for protocol V2
>> +void lazy_load_hidden_refs(void) {
>> +	lazy_load_hidden = 1;
>> +}
>> +
> 
> What does lazy_load_hidden do?
> 
> I know this is a lot to go thru for your first patch series, but please
> don't get discouraged! Feel free to ask any questions if you're confused
> about any of the feedback. We didn't dive too deeply into the specifics
> of your code since we believe there are higher level fundamental issues
> you should address first. There has also been similar discussion regarding
> differing ACLs within a single repository so it is probably worth a read
> here[1].

I will not be discouraged, and I’m very glad and appreciate that I can receive
important review notes from Junio and the mailing list. I want to do more
contributions to git and wish one day I can help to review other patches. But
first I will fix my patches and make it more clearly.

I need to reply first and wishing not making noise, because I think it will
take me couple days to resolve the review comments and update the patches again.

Thanks again.

> 
> [1] <CAJoAoZmsuwYCA8XGziEA-qwghg9h22Af98JQE1AuHHBRfQgrDA@mail.gmail.com>
> 

[PATCH v4 2/3] t1419: add test cases for hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'hide-refs' hook which is used to
filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 t/t1419-hide-refs-hook.sh                     | 142 +++++++++++++++
 t/t1419/common-functions.sh                   |  80 +++++++++
 t/t1419/once-0000-abnormal-hide-refs-hook.sh  | 161 ++++++++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  77 +++++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh | 122 +++++++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  87 ++++++++++
 6 files changed, 669 insertions(+)
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/once-0000-abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh

diff --git a/t/t1419-hide-refs-hook.sh b/t/t1419-hide-refs-hook.sh
new file mode 100755
index 00000000000..52ce61c0e1f
--- /dev/null
+++ b/t/t1419-hide-refs-hook.sh
@@ -0,0 +1,142 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test hide-refs hook'
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+. ./test-lib.sh
+
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+setup_bare_repo_and_work_repo () {
+	# Refs of bare_repo : main(A)
+	# Refs of work_repo: main(A)  tags/v123
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git bare_repo.git.dump &&
+		rm -rf work_repo work_repo.dump &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git remote add origin ../bare_repo.git &&
+			git update-ref refs/heads/dev $A &&
+			git update-ref refs/heads/main $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push origin +refs/heads/*:refs/heads/* &&
+			git push origin +refs/tags/*:refs/tags/* &&
+			git push origin +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# setup pre-receive hook
+		write_script bare_repo.git/hooks/pre-receive <<-\EOF &&
+		exec >&2
+		echo "# pre-receive hook"
+		while read old new ref
+		do
+			echo "pre-receive< $old $new $ref"
+		done
+		EOF
+
+		# setup update hook
+		write_script bare_repo.git/hooks/update <<-\EOF &&
+		exec >&2
+		echo "# update hook"
+		echo "update< $@"
+		EOF
+
+		# setup post-receive hook
+		write_script bare_repo.git/hooks/post-receive <<-\EOF
+		exec >&2
+		echo "# post-receive hook"
+		while read old new ref
+		do
+			echo "post-receive< $old $new $ref"
+		done
+		EOF
+	'
+}
+
+run_hide_refs_hook_tests() {
+	case $1 in
+		http)
+			PROTOCOL="HTTP protocol"
+			BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			BAREREPO_PREFIX="$HTTPD_URL"/smart
+			;;
+		local)
+			PROTOCOL="builtin protocol"
+			BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+			BAREREPO_PREFIX="$(pwd)"
+			;;
+	esac
+
+	BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+	GIT_TEST_PROTOCOL_VERSION=$2
+
+	# Run test cases for 'hide-refs' hook
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		# Initialize the bare_repo repository and work_repo
+		setup_bare_repo_and_work_repo
+		git -C work_repo remote set-url origin "$BAREREPO_URL"
+		cp -rf work_repo work_repo.dump
+
+		git -C bare_repo.git config --local http.receivepack true
+		git -C bare_repo.git config --add transfer.hiderefs force:HEAD
+		git -C bare_repo.git config --add transfer.hiderefs force:refs
+		cp -rf bare_repo.git bare_repo.git.dump
+
+		if test "$1" = "http"; then
+			setup_askpass_helper
+			rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+			mv bare_repo.git "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+		fi
+
+		. "$t"
+	done
+}
+
+
+setup_bare_repo_and_work_repo
+BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+BAREREPO_PREFIX="$(pwd)"
+BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+
+# Load test cases that only need to be executed once.
+for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
+do
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:HEAD
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:refs
+	. "$t"
+done
+
+for protocol in 1 2
+do
+	# Run test cases for 'hide-refs' hook on local file protocol.
+	run_hide_refs_hook_tests local $protocol
+done
+
+ROOT_PATH="$PWD"
+. "$TEST_DIRECTORY"/lib-gpg.sh
+. "$TEST_DIRECTORY"/lib-httpd.sh
+. "$TEST_DIRECTORY"/lib-terminal.sh
+
+start_httpd
+set_askpass user@host pass@host
+
+# Run test cases for 'hide-refs' hook on HTTP protocol.
+for protocol in 1 2
+do
+	run_hide_refs_hook_tests http $protocol
+done
+
+test_done
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..e86c3d11317
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,80 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-push, git-show-ref and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.  We also
+# remove some locale error messages. The emitted human-readable errors are
+# redundant to the more machine-readable output the tests already assert.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$TRASH_DIRECTORY/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
+}
+
+filter_out_hide_refs_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+filter_out_user_friendly_and_stable_output () {
+	make_user_friendly_and_stable_output |
+		sed -n ${1+"$@"}
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/once-0000-abnormal-hide-refs-hook.sh b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
new file mode 100644
index 00000000000..c4ff642264e
--- /dev/null
+++ b/t/t1419/once-0000-abnormal-hide-refs-hook.sh
@@ -0,0 +1,161 @@
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs \
+			--die-read-version \
+			-r refs/heads/main
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when write version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-write-version
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when write version" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-write-version option
+		fatal: can not read version message from hook hide-refs
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read first filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-first-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-first-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read second filter request" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-read-second-ref
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-read-second-ref option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die while filtring refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+'
+
+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		fatal: hook hide-refs died abnormally
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..1766d869d58
--- /dev/null
+++ b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
@@ -0,0 +1,77 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide all refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide branches" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide pull refs and tags" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-B>	HEAD
+		<COMMIT-A>	refs/heads/dev
+		<COMMIT-B>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..d2fc5f0c4f4
--- /dev/null
+++ b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
@@ -0,0 +1,122 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide no refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide all refs" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	test_must_fail git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which some branches" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is not hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $A
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide pull refs and tags" '
+	rm -rf local.git &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> refs/heads/dev
+		<COMMIT-B> refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is hidden" '
+	rm -rf local.git &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $C
+'
diff --git a/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..198fbf5fa5c
--- /dev/null
+++ b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
@@ -0,0 +1,87 @@
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+		test-tool hide-refs
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide no refs" '
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
+		To <URL/of/bare_repo.git>
+		 * [new branch]      HEAD -> new
+		EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while hide-refs hook hide no refs" '
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		remote: # update hook        Z
+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
+		remote: # post-receive hook        Z
+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
+		To <URL/of/bare_repo.git>
+		 - [deleted]         dev
+	EOF
+	test_cmp expect actual
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
+	rm -rf work_repo &&
+	cp -rf work_repo.dump work_repo &&
+	rm -rf "$BAREREPO_GIT_DIR" &&
+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
+	test-tool hide-refs \
+		-r "HEAD" \
+		-r "refs/heads/dev" \
+		-r "refs/heads/main" \
+		-r "refs/pull-requests/1/head" \
+		-r "refs/tags/v123"
+	EOF
+'
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide all refs" '
+	create_commits_in work_repo E &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		remote: # pre-receive hook        Z
+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH v4 3/3] doc: add documentation for the hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <16657101987@163.com>

"git upload-pack" or "git receive-pack" can use "hide-refs"
hook to filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..456998bba56 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[hide-refs]]
+hide-refs
+~~~~~~~~~
+
+This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
+during the reference discovery phase, each reference and will be filtered
+by this hook. The hook executes once with no arguments for each
+'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
+a version number and server process name ('uploadpack' or 'receive') will
+send to it in pkt-line format, followed by a flush-pkt. The hook should
+respond with its version number.
+
+During reference discovery phase, each reference will be filtered by this
+hook. In the following example, the letter 'G' stands for 'git-receive-pack'
+or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
+decides if the reference will be hidden or not, it sends result back in
+pkt-line format protocol, a response "hide" the references will hide
+to the client and can not fetch it even in protocol V2.
+
+	# Version negotiation
+	G: PKT-LINE(version=1\0uploadpack)
+	G: flush-pkt
+	H: PKT-LINE(version=1)
+	H: flush-pkt
+
+	# Send reference filter request to hook
+	G: PKT-LINE(ref <refname>:<refnamefull>)
+	G: flush-pkt
+
+	# Receive result from the hook.
+	# Case 1: this reference is hidden
+	H: PKT-LINE(hide)
+	H: flush-pkt
+
+	# Case 2: this reference can be advertised
+	H: flush-pkt
+
+To enable the `hide-refs` hook, we should config hiderefs with `force:`
+option, eg:
+
+	git config --add transfer.hiderefs force:refs/prefix1/
+	git config --add uploadpack.hiderefs force:!refs/prefix2/
+
+the `hide-refs` will be called during reference discovery phase and
+check each matched reference, a 'hide' response means the reference will
+be hidden for its private data even if `allowTipSHA1InWant` and
+`allowReachableSHA1InWant` are set to true.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget

[PATCH v5 0/5] hiderefs: add hide-refs hook to hide refs dynamically

[Sun Chao via GitGitGadget]

Hi, thanks for reviewing the patches, we'd like to add a new hook called
"hide-refs", which can be used to hide refs dynamically according to the
user's access to the git servers. Here is the reasons:

(1) There are lots of development teams in our company (Huawei), and they
have been using the Gerrit platform during development work for a long time
and get used to its features, e.g. Gerrit's reference-level control feature
can be used to arrange different small teams to develop on different
branches, some branches will be only visible to special teams, this way of
working effectively shields the interference between different branches.

In this way Gerrit could protect each teams' codes privately until
administrators decided to merge or cherry-pick the codes together. Most of
the developers cannot see all the references of the repositories, and Gerrit
does not apply the 'fork' feature which exists in Github or Gitlab, if you
want to create a code review task you can just run git command like:

git push origin HEAD:refs/for/main


Developers cannot fork their own repositories but clone from and push to the
same repositories, so in Huawei, when we talk about Gerrit, we always treat
it as a centralized workflow platform and lots of teams like its features.

(2) Gerrit implement the JGit which supports the wire git protocol V1 and
V2. We have built our new git server based on Gitlab-CE
[https://gitlab.com/rluna-gitlab/gitlab-ce] and it uses CGit (Compared to
Gerrit's JGit, someone call the Git itself as CGit). Recently we were
planning to migrate some development teams from Gerrit platform to the new
server. So we wish the new server can work like Gerrit which allow users
create code review task by git push command and also can hide refs according
to the user's access.

Commit 15d3af5e22 (receive-pack: add new proc-receive hook, 2020-08-27) make
it possible to create a code review by a single proc-receive command on the
server side. What we need right now is reference-level permission control,
we referred to Gerrit's implementation mechanism and added a new reference
filtering mechanism:

a) We config the reference level control rules on server side, eg:

    ```json
    {
        {
            "ref":"refs/heads/stable/*",
            "action":"read",
            "access":"allow",
            "user_group":"dev_group_1"
        },
        {
            "ref":"refs/heads/stable/*",
            "action":"read",
            "access":"deny",
            "user_group":"dev_group_1"
        },
        {
            "ref":"refs/for/*",
            "action":"create-codereview",
            "access":"access",
            "user_group":"dev_group_1"
        },
        ... ...
    }
    ```

b) During upload-pack and receive-pack the hide-refs hook will hide the refs
    that the user cannot read during reference advertise phase according to
    the reference level control rules above.
c) We also add a `HIDDEN_REF_FORCE` flag to make sure the private data
    of these hide refs cannot be fetched by the git clients.


(3) When we talk about permission control in git, it may comes to the
directory permission control, but however we can use the workflow like
Gerrit to control the contents of a repository by put its parts to different
branches, and merge or cherry-pick them together if needed.

In special scenarios, we need to protect the content on some branches from
being seen by others and leaked to other teams, now we can achieve that by
the new "hide-refs" hook.

And here I will talk about how the "hide-refs" works on server side:

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered with
this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client.

# Version negotiation
G: PKT-LINE(version=1\0uploadpack)
G: flush-pkt
H: PKT-LINE(version=1)
H: flush-pkt

# Send reference filter request to hook
G: PKT-LINE(ref <refname>:<refname_full>)
G: flush-pkt

# Receive result from the hook.
# Case 1: this reference is hidden
H: PKT-LINE(hide)
H: flush-pkt

# Case 2: this reference can be advertised
H: flush-pkt


To enable the hide-refs hook, we should config hiderefs with hook: option,
eg:

git config --add transfer.hiderefs hook:refs/prefix1/
git config --add uploadpack.hiderefs hook:!refs/prefix2/


the hide-refs will be called during reference discovery phase and check each
matched reference, a 'hide' response means the reference will be hidden for
its private data even if allowTipSHA1InWant or allowReachableSHA1InWant are
set to true.

Sun Chao (5):
  hiderefs: add hide-refs hook to hide refs dynamically
  hiderefs: use new flag to mark force hidden refs
  hiderefs: hornor hide flags in wire protocol V2
  test: add test cases for hide-refs hook
  doc: add documentation for the hide-refs hook

 Documentation/githooks.txt                    |  48 +++
 Makefile                                      |   1 +
 ls-refs.c                                     |   2 +-
 refs.c                                        | 288 +++++++++++++++++-
 refs.h                                        |   7 +
 t/helper/test-hide-refs.c                     | 107 +++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 112 +++++++
 t/t1419/abnormal-hide-refs-hook.sh            |  82 +++++
 t/t1419/common-functions.sh                   |  74 +++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  45 +++
 ...st-0002-upload-pack-with-hide-refs-hook.sh |  47 +++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  42 +++
 upload-pack.c                                 |  25 +-
 15 files changed, 855 insertions(+), 27 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh


base-commit: 79f2338b3746d23454308648b2491e5beba4beff
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v5
Pull-Request: https://github.com/git/git/pull/1301

Range-diff vs v4:

 1:  01c63ea5fee ! 1:  278bd185aec hide-refs: add hook to force hide refs
     @@ Metadata
      Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
     -    hide-refs: add hook to force hide refs
     +    hiderefs: add hide-refs hook to hide refs dynamically
      
          Gerrit is implemented by JGit and is known as a centralized workflow system
          which supports reference-level access control for repository. If we choose
     @@ Commit message
          hook. In the following example, the letter 'G' stands for 'git-receive-pack'
          or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
          decides if the reference will be hidden or not, it sends result back in
     -    pkt-line format protocol, a response "hide" means the references will hide
     -    to the client and can not fetch its private data even in protocol V2.
     +    pkt-line format protocol, a response "hide" means the references will be
     +    hidden to the client.
      
                  # Version negotiation
                  G: PKT-LINE(version=1\0uploadpack)
     @@ Commit message
                  # Case 2: this reference can be advertised
                  H: flush-pkt
      
     -    To enable the `hide-refs` hook, we should config hiderefs with `force:`
     +    To enable the `hide-refs` hook, we should config hiderefs with `hook:`
          option, eg:
      
     -            git config --add transfer.hiderefs force:refs/prefix1/
     -            git config --add uploadpack.hiderefs force:!refs/prefix2/
     -
     -    the `hide-refs` will be called during reference discovery phase and
     -    check each matched reference, a 'hide' response means the reference will
     -    be hidden for its private data even if `allowTipSHA1InWant` or
     -    `allowReachableSHA1InWant` are set to true.
     +            git config --add transfer.hiderefs hook:refs/prefix1/
     +            git config --add uploadpack.hiderefs hook:!refs/prefix2/
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     - ## Makefile ##
     -@@ Makefile: TEST_BUILTINS_OBJS += test-wildmatch.o
     - TEST_BUILTINS_OBJS += test-windows-named-pipe.o
     - TEST_BUILTINS_OBJS += test-write-cache.o
     - TEST_BUILTINS_OBJS += test-xml-encode.o
     -+TEST_BUILTINS_OBJS += test-hide-refs.o
     - 
     - # Do not add more tests here unless they have extra dependencies. Add
     - # them in TEST_BUILTINS_OBJS above.
     -
     - ## builtin/receive-pack.c ##
     -@@ builtin/receive-pack.c: static int show_ref_cb(const char *path_full, const struct object_id *oid,
     - 	struct oidset *seen = data;
     - 	const char *path = strip_namespace(path_full);
     - 
     --	if (ref_is_hidden(path, path_full))
     -+	if (ref_is_hidden(path, path_full) || ref_is_force_hidden(path, path_full))
     - 		return 0;
     - 
     - 	/*
     -@@ builtin/receive-pack.c: static void reject_updates_to_hidden(struct command *commands)
     - 		strbuf_setlen(&refname_full, prefix_len);
     - 		strbuf_addstr(&refname_full, cmd->ref_name);
     - 
     --		if (!ref_is_hidden(cmd->ref_name, refname_full.buf))
     -+		if (!ref_is_hidden(cmd->ref_name, refname_full.buf) &&
     -+			!ref_is_force_hidden(cmd->ref_name, refname_full.buf))
     - 			continue;
     - 		if (is_null_oid(&cmd->new_oid))
     - 			cmd->error_string = "deny deleting a hidden ref";
     -
     - ## ls-refs.c ##
     -@@ ls-refs.c: static int send_ref(const char *refname, const struct object_id *oid,
     - 
     - 	strbuf_reset(&data->buf);
     - 
     --	if (ref_is_hidden(refname_nons, refname))
     -+	if (mark_our_ref(refname_nons, refname, oid))
     - 		return 0;
     - 
     - 	if (!ref_match(&data->prefixes, refname_nons))
     -
       ## refs.c ##
      @@
       #include "lockfile.h"
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
       }
       
       static struct string_list *hide_refs;
     --
     -+static struct string_list *force_hide_refs;
     ++static struct string_list *hook_hide_refs;
      +static struct strbuf hide_refs_section = STRBUF_INIT;
     + 
       int parse_hide_refs_config(const char *var, const char *value, const char *section)
       {
       	const char *key;
     -+	int force = 0;
     ++	int hook = 0;
      +
       	if (!strcmp("transfer.hiderefs", var) ||
       	    (!parse_config_key(var, section, NULL, NULL, &key) &&
       	     !strcmp(key, "hiderefs"))) {
     - 		char *ref;
     - 		int len;
     -+		int forcelen;
     +@@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const char *secti
       
       		if (!value)
       			return config_error_nonbool(var);
      +
     -+		forcelen = strlen("force:");
     -+		len = strlen(value);
     -+		if ((len >= forcelen) && !strncmp(value, "force:", forcelen)) {
     -+			if (len == forcelen)
     -+				return error(_("missing value for '%s' with force option"), var);
     -+
     -+			force = 1;
     -+			value += forcelen;
     ++		/*
     ++		 * the prefix 'hook:' means that the matched refs will be
     ++		 * checked by the hide-refs hook dynamically, we need to put
     ++		 * the 'ref' string to the hook_hide_refs list
     ++		 */
     ++		if (skip_prefix(value, "hook:", &value)) {
     ++			if (!strlen(value))
     ++				return error(_("missing value for '%s' after hook option"), var);
     ++			hook = 1;
      +		}
      +
       		ref = xstrdup(value);
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      -			CALLOC_ARRAY(hide_refs, 1);
      -			hide_refs->strdup_strings = 1;
      +
     -+		if (force) {
     -+			if (!force_hide_refs) {
     -+				CALLOC_ARRAY(force_hide_refs, 1);
     -+				force_hide_refs->strdup_strings = 1;
     ++		if (hook) {
     ++			if (!hook_hide_refs) {
     ++				CALLOC_ARRAY(hook_hide_refs, 1);
     ++				hook_hide_refs->strdup_strings = 1;
      +			}
     -+			string_list_append(force_hide_refs, ref);
     ++			string_list_append(hook_hide_refs, ref);
      +		} else {
      +			if (!hide_refs) {
      +				CALLOC_ARRAY(hide_refs, 1);
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      -		string_list_append(hide_refs, ref);
       	}
      +
     -+	if (hide_refs_section.len == 0) {
     ++	/*
     ++	 * Once hide-refs hook is invoked, Git need to do version negotiation,
     ++	 * with it, version number and process name ('uploadpack' or 'receive')
     ++	 * will send to it in pkt-line format, the proccess name is recorded
     ++	 * by hide_refs_section
     ++	 */
     ++	if (hook && hide_refs_section.len == 0)
      +		strbuf_addstr(&hide_refs_section, section);
     -+	}
      +
       	return 0;
       }
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      -int ref_is_hidden(const char *refname, const char *refname_full)
      +static struct child_process *hide_refs_proc;
      +static struct packet_reader *hide_refs_reader;
     -+static void create_hide_refs_process(void) {
     ++
     ++/*
     ++ * Create the hide-refs hook child process and complete version negotiation,
     ++ * return non-zero upon success, otherwise 0
     ++ */
     ++static int create_hide_refs_process(void)
     ++{
      +	struct child_process *proc;
      +	struct packet_reader *reader;
      +	const char *hook_path;
      +	int version = 0;
     -+	int code;
     ++	int err;
      +
      +	hook_path = find_hook("hide-refs");
     -+	if (!hook_path) {
     -+		die("can not find hide-refs hook");
     -+	}
     ++	if (!hook_path)
     ++		return 0;
      +
     -+	proc = (struct child_process *) xcalloc (1, sizeof (struct child_process));
     -+	reader = (struct packet_reader *) xcalloc (1, sizeof(struct packet_reader));
     ++	proc = (struct child_process *)xcalloc(1, sizeof (struct child_process));
     ++	reader = (struct packet_reader *)xcalloc(1, sizeof(struct packet_reader));
      +
      +	child_process_init(proc);
      +	strvec_push(&proc->args, hook_path);
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +	proc->trace2_hook_name = "hide-refs";
      +	proc->err = 0;
      +
     -+	code = start_command(proc);
     -+	if (code)
     -+		die("can not run hook hide-refs");
     ++	err = start_command(proc);
     ++	if (err)
     ++		goto cleanup;
      +
      +	sigchain_push(SIGPIPE, SIG_IGN);
      +
      +	/* Version negotiaton */
      +	packet_reader_init(reader, proc->out, NULL, 0,
     -+			   PACKET_READ_CHOMP_NEWLINE |
     -+			   PACKET_READ_GENTLE_ON_EOF);
     -+	code = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
     -+	if (!code)
     -+		code = packet_flush_gently(proc->in);
     ++			   PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
     ++	err = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
     ++	if (!err)
     ++		err = packet_flush_gently(proc->in);
      +
     -+	if (!code)
     ++	if (!err)
      +		for (;;) {
      +			enum packet_read_status status;
      +
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +			if (status != PACKET_READ_NORMAL) {
      +				/* Check whether hide-refs exited abnormally */
      +				if (status == PACKET_READ_EOF)
     -+					die("can not read version message from hook hide-refs");
     ++					goto failure;
      +				break;
      +			}
      +
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +			}
      +		}
      +
     -+	if (code)
     -+		die("can not read version message from hook hide-refs");
     ++	if (err)
     ++		goto failure;
      +
      +	switch (version) {
      +	case 0:
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +	case 1:
      +		break;
      +	default:
     -+		die(_("hook hide-refs version '%d' is not supported"), version);
     ++		trace_printf(_("hook hide-refs version '%d' is not supported"), version);
     ++		goto failure;
      +	}
      +
      +	sigchain_pop(SIGPIPE);
      +
      +	hide_refs_proc = proc;
      +	hide_refs_reader = reader;
     -+	return;
     ++	return 1;
     ++
     ++failure:
     ++	close(proc->in);
     ++	close(proc->out);
     ++	kill(proc->pid, SIGTERM);
     ++	finish_command_in_signal(proc);
     ++
     ++cleanup:
     ++	free(proc);
     ++	free(reader);
     ++	sigchain_pop(SIGPIPE);
     ++	return 0;
      +}
      +
     -+static int ref_force_hidden_check(const char *refname, const char *refname_full)
     ++/* If hide-refs child process start failed, set skip_hide_refs_proc to true */
     ++static int skip_hide_refs_proc;
     ++
     ++/*
     ++ * Return non-zero if hide-refs hook want to hide the ref and 0 otherwise,
     ++ * and return 0 if hide-refs child proccess start failed or exit abnormally
     ++ */
     ++static int ref_hidden_check_by_hook(const char *refname, const char *refname_full)
      +{
      +	struct strbuf buf = STRBUF_INIT;
     -+	int code;
     ++	int err;
      +	int ret = 0;
      +
     -+	if (!force_hide_refs) {
     ++	if (skip_hide_refs_proc)
      +		return 0;
     -+	}
      +
     -+	if (!hide_refs_proc) {
     -+		create_hide_refs_process();
     -+	}
     ++	if (!hide_refs_proc)
     ++		if (!create_hide_refs_process()) {
     ++			skip_hide_refs_proc = 1;
     ++			return 0;
     ++		}
      +
      +	sigchain_push(SIGPIPE, SIG_IGN);
     -+	code = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
     -+	if (code)
     -+		die("hook hide-refs died abnormally");
     ++	err = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
     ++	if (err)
     ++		goto cleanup;
      +
     -+	code = packet_flush_gently(hide_refs_proc->in);
     -+	if (code)
     -+		die("hook hide-refs died abnormally");
     ++	err = packet_flush_gently(hide_refs_proc->in);
     ++	if (err)
     ++		goto cleanup;
      +
      +	for (;;) {
      +		enum packet_read_status status;
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +		if (status != PACKET_READ_NORMAL) {
      +			/* Check whether hide-refs exited abnormally */
      +			if (status == PACKET_READ_EOF)
     -+				die("hook hide-refs died abnormally");
     ++				goto cleanup;
      +			break;
      +		}
      +
     -+		strbuf_reset(&buf);
      +		strbuf_addstr(&buf, hide_refs_reader->line);
      +	}
      +
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
      +
      +	sigchain_pop(SIGPIPE);
      +	return ret;
     ++
     ++cleanup:
     ++	close(hide_refs_proc->in);
     ++	close(hide_refs_proc->out);
     ++	kill(hide_refs_proc->pid, SIGTERM);
     ++	finish_command_in_signal(hide_refs_proc);
     ++
     ++	free(hide_refs_proc);
     ++	free(hide_refs_reader);
     ++	sigchain_pop(SIGPIPE);
     ++
     ++	skip_hide_refs_proc = 1;
     ++	return 0;
      +}
      +
     -+static int ref_hidden_check(const char *refname, const char *refname_full, int force)
     ++static int ref_hidden_check(const char *refname, const char *refname_full, int hook)
       {
      +	struct string_list *hide_refs_list = hide_refs;
       	int i;
       
      -	if (!hide_refs)
     -+	if (force)
     -+		hide_refs_list = force_hide_refs;
     ++	if (hook)
     ++		hide_refs_list = hook_hide_refs;
      +
      +	if (!hide_refs_list)
       		return 0;
     @@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
      +		    (!*p || *p == '/')) {
      +			if (neg)
      +				return 0;
     -+			if (!force)
     ++			if (!hook)
      +				return 1;
     -+			return ref_force_hidden_check(refname, refname_full);
     ++			return ref_hidden_check_by_hook(refname, refname_full);
      +		}
       	}
       	return 0;
     @@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
       
      +int ref_is_hidden(const char *refname, const char *refname_full)
      +{
     -+	return ref_hidden_check(refname, refname_full, 0);
     -+}
     -+
     -+int ref_is_force_hidden(const char *refname, const char *refname_full)
     -+{
     -+	return ref_hidden_check(refname, refname_full, 1);
     -+}
     -+
     -+#define OUR_REF		(1u << 12)
     -+#define HIDDEN_REF	(1u << 19)
     -+#define HIDDEN_REF_FORCE	(1u << 20)
     -+static int has_force_hidden;
     -+int mark_our_ref(const char *refname, const char *refname_full,
     -+			const struct object_id *oid)
     -+{
     -+	struct object *o;
     -+
     -+	if (!oid || is_null_oid(oid)) {
     -+		return 0;
     -+	}
     -+
     -+	o = lookup_unknown_object(the_repository, oid);
     -+	if (ref_is_force_hidden(refname, refname_full)) {
     -+		o->flags |= HIDDEN_REF_FORCE;
     -+		has_force_hidden = 1;
     -+		return 1;
     -+	}
     -+	if (ref_is_hidden(refname, refname_full)) {
     -+		o->flags |= HIDDEN_REF;
     ++	if (ref_hidden_check(refname, refname_full, 0) ||
     ++	    ref_hidden_check(refname, refname_full, 1))
      +		return 1;
     -+	}
     -+	o->flags |= OUR_REF;
      +	return 0;
      +}
     -+
     -+int has_force_hidden_refs(void) {
     -+	return has_force_hidden;
     -+}
      +
       const char *find_descendant_ref(const char *dirname,
       				const struct string_list *extras,
       				const struct string_list *skip)
     -
     - ## refs.h ##
     -@@ refs.h: int parse_hide_refs_config(const char *var, const char *value, const char *);
     -  * parameter always points to the full ref name.
     -  */
     - int ref_is_hidden(const char *, const char *);
     -+int ref_is_force_hidden(const char *, const char *);
     -+/* return non-zero if the ref is hidden, otherwise 0 */
     -+int mark_our_ref(const char *refname, const char *refname_full,
     -+			const struct object_id *oid);
     -+int has_force_hidden_refs(void);
     -+void lazy_load_hidden_refs(void);
     - 
     - enum ref_type {
     - 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
     -
     - ## serve.c ##
     -@@
     - #include "cache.h"
     - #include "repository.h"
     - #include "config.h"
     -+#include "refs.h"
     - #include "pkt-line.h"
     - #include "version.h"
     - #include "ls-refs.h"
     -@@ serve.c: void protocol_v2_serve_loop(int stateless_rpc)
     - 	 * a single request/response exchange
     - 	 */
     - 	if (stateless_rpc) {
     -+		lazy_load_hidden_refs();
     - 		process_request();
     - 	} else {
     - 		for (;;)
     -
     - ## t/helper/test-hide-refs.c (new) ##
     -@@
     -+#include "cache.h"
     -+#include "hash.h"
     -+#include "config.h"
     -+#include "connect.h"
     -+#include "parse-options.h"
     -+#include "pkt-line.h"
     -+#include "sigchain.h"
     -+#include "test-tool.h"
     -+
     -+static const char *hide_refs_usage[] = {
     -+	"test-tool hide-refs [<options>...]",
     -+	NULL
     -+};
     -+
     -+static int die_read_version;
     -+static int die_write_version;
     -+static int die_read_first_ref;
     -+static int die_read_second_ref;
     -+static int die_after_proc_ref;
     -+static int verbose;
     -+static int version = 1;
     -+static int first_ref;
     -+static int second_ref;
     -+static int hash_size = GIT_SHA1_HEXSZ;
     -+static struct string_list returns = STRING_LIST_INIT_NODUP;
     -+
     -+struct command {
     -+	struct command *next;
     -+	const char *error_string;
     -+	unsigned int skip_update:1,
     -+		     did_not_exist:1;
     -+	int index;
     -+	struct object_id old_oid;
     -+	struct object_id new_oid;
     -+	char ref_name[FLEX_ARRAY]; /* more */
     -+};
     -+
     -+static void hide_refs_verison(struct packet_reader *reader) {
     -+	int server_version = 0;
     -+
     -+	if (die_read_version)
     -+		die("die with the --die-read-version option");
     -+
     -+	for (;;) {
     -+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
     -+			break;
     -+
     -+		/* Ignore version negotiation for version 0 */
     -+		if (version == 0)
     -+			continue;
     -+
     -+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
     -+			server_version = atoi(reader->line+8);
     -+			if (server_version != 1)
     -+				die("bad protocol version: %d", server_version);
     -+		}
     -+	}
     -+
     -+	if (die_write_version)
     -+		die("die with the --die-write-version option");
     -+
     -+	packet_write_fmt(1, "version=%d\n", version);
     -+	packet_flush(1);
     -+}
     -+
     -+static void hide_refs_proc(struct packet_reader *reader)
     -+{
     -+	const char *p;
     -+	struct strbuf buf = STRBUF_INIT;
     -+	enum packet_read_status status;
     -+
     -+	if (!first_ref) {
     -+		if (die_read_first_ref)
     -+			die("die with the --die-read-first-ref option");
     -+
     -+		first_ref = 1;
     -+	}
     -+
     -+	if (first_ref && !second_ref) {
     -+		if (die_read_second_ref)
     -+			die("die with the --die-read-second-ref option");
     -+
     -+		second_ref = 1;
     -+	}
     -+
     -+	for (;;) {
     -+		status = packet_reader_read(reader);
     -+		if (status == PACKET_READ_EOF)
     -+			exit(0);
     -+
     -+		if (status != PACKET_READ_NORMAL)
     -+			break;
     -+
     -+		p = reader->line;
     -+		strbuf_reset(&buf);
     -+		strbuf_addstr(&buf, reader->line);
     -+	}
     -+
     -+	p = strchr(buf.buf, ':');
     -+	if (unsorted_string_list_has_string(&returns, p + 1)) {
     -+		packet_write_fmt(1, "hide");
     -+	}
     -+
     -+	if (die_after_proc_ref)
     -+		die("die with the --die-after-proc-refs option");
     -+
     -+	packet_flush(1);
     -+}
     -+
     -+int cmd__hide_refs(int argc, const char **argv) {
     -+	int nongit_ok = 0;
     -+	struct packet_reader reader;
     -+	const char *value = NULL;
     -+	struct option options[] = {
     -+		OPT_BOOL(0, "die-read-version", &die_read_version,
     -+			 "die when reading version"),
     -+		OPT_BOOL(0, "die-write-version", &die_write_version,
     -+			 "die when writing version"),
     -+		OPT_BOOL(0, "die-read-first-ref", &die_read_first_ref,
     -+			 "die when reading first reference"),
     -+		OPT_BOOL(0, "die-read-second-ref", &die_read_second_ref,
     -+			 "die when reading second reference"),
     -+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
     -+			 "die after proc ref"),
     -+		OPT_STRING_LIST('r', "reserved", &returns, "refs-to-force-hidden",
     -+				"refs that will force hide"),
     -+		OPT__VERBOSE(&verbose, "be verbose"),
     -+		OPT_INTEGER('V', "version", &version,
     -+			    "use this protocol version number"),
     -+		OPT_END()
     -+	};
     -+
     -+	setup_git_directory_gently(&nongit_ok);
     -+
     -+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
     -+	if (argc > 0)
     -+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
     -+
     -+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
     -+
     -+	if (!git_config_get_value("extensions.objectformat", &value)) {
     -+		if (!strcmp(value, "sha256"))
     -+			hash_size = GIT_SHA256_HEXSZ;
     -+	}
     -+
     -+	hide_refs_verison(&reader);
     -+	for (;;) {
     -+		hide_refs_proc(&reader);
     -+	}
     -+
     -+	return 0;
     -+}
     -
     - ## t/helper/test-tool.c ##
     -@@ t/helper/test-tool.c: static struct test_cmd cmds[] = {
     - 	{ "regex", cmd__regex },
     - 	{ "repository", cmd__repository },
     - 	{ "revision-walking", cmd__revision_walking },
     -+	{ "hide-refs", cmd__hide_refs },
     - 	{ "run-command", cmd__run_command },
     - 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
     - 	{ "serve-v2", cmd__serve_v2 },
     -
     - ## t/helper/test-tool.h ##
     -@@ t/helper/test-tool.h: int cmd__reftable(int argc, const char **argv);
     - int cmd__regex(int argc, const char **argv);
     - int cmd__repository(int argc, const char **argv);
     - int cmd__revision_walking(int argc, const char **argv);
     -+int cmd__hide_refs(int argc, const char **argv);
     - int cmd__run_command(int argc, const char **argv);
     - int cmd__scrap_cache_tree(int argc, const char **argv);
     - int cmd__serve_v2(int argc, const char **argv);
     -
     - ## upload-pack.c ##
     -@@
     - #define NOT_SHALLOW	(1u << 17)
     - #define CLIENT_SHALLOW	(1u << 18)
     - #define HIDDEN_REF	(1u << 19)
     -+#define HIDDEN_REF_FORCE	(1u << 20)
     - 
     --#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
     --		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
     -+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
     -+		NOT_SHALLOW | CLIENT_SHALLOW)
     - 
     - /* Enum for allowed unadvertised object request (UOR) */
     - enum allow_uor {
     -@@ upload-pack.c: static void receive_needs(struct upload_pack_data *data,
     - 		packet_flush(1);
     - }
     - 
     --/* return non-zero if the ref is hidden, otherwise 0 */
     --static int mark_our_ref(const char *refname, const char *refname_full,
     --			const struct object_id *oid)
     --{
     --	struct object *o = lookup_unknown_object(the_repository, oid);
     --
     --	if (ref_is_hidden(refname, refname_full)) {
     --		o->flags |= HIDDEN_REF;
     --		return 1;
     --	}
     --	o->flags |= OUR_REF;
     --	return 0;
     --}
     --
     - static int check_ref(const char *refname_full, const struct object_id *oid,
     - 		     int flag, void *cb_data)
     - {
     -@@ upload-pack.c: static int parse_want_ref(struct packet_writer *writer, const char *line,
     - 
     - 		strbuf_addf(&refname, "%s%s", get_git_namespace(), refname_nons);
     - 		if (ref_is_hidden(refname_nons, refname.buf) ||
     -+			ref_is_force_hidden(refname_nons, refname.buf) ||
     - 		    read_ref(refname.buf, &oid)) {
     - 			packet_writer_error(writer, "unknown ref %s", refname_nons);
     - 			die("unknown ref %s", refname_nons);
     -@@ upload-pack.c: enum fetch_state {
     - 	FETCH_DONE,
     - };
     - 
     -+static int lazy_load_hidden = 0;
     -+// lazy load hidden refs for protocol V2
     -+void lazy_load_hidden_refs(void) {
     -+	lazy_load_hidden = 1;
     -+}
     -+
     - int upload_pack_v2(struct repository *r, struct packet_reader *request)
     - {
     - 	enum fetch_state state = FETCH_PROCESS_ARGS;
     -@@ upload-pack.c: int upload_pack_v2(struct repository *r, struct packet_reader *request)
     - 				state = FETCH_DONE;
     - 			break;
     - 		case FETCH_SEND_PACK:
     -+			if (lazy_load_hidden) {
     -+				head_ref_namespaced(check_ref, NULL);
     -+				for_each_namespaced_ref(check_ref, NULL);
     -+			}
     -+			if (has_force_hidden_refs())
     -+				check_non_tip(&data);
     - 			send_wanted_ref_info(&data);
     - 			send_shallow_info(&data);
     - 
     -
     - ## upload-pack.h ##
     -@@ upload-pack.h: struct strbuf;
     - int upload_pack_advertise(struct repository *r,
     - 			  struct strbuf *value);
     - 
     -+void lazy_load_hidden_refs(void);
     - #endif /* UPLOAD_PACK_H */
 -:  ----------- > 2:  0df5ecc216d hiderefs: use new flag to mark force hidden refs
 -:  ----------- > 3:  de73f5a6fd9 hiderefs: hornor hide flags in wire protocol V2
 2:  b8a490cb3df ! 4:  fb135cb3bd3 t1419: add test cases for hide-refs hook
     @@ Metadata
      Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
     -    t1419: add test cases for hide-refs hook
     +    test: add test cases for hide-refs hook
      
          Add test cases for the new 'hide-refs' hook which is used to
          filter the references during reference discovery phase.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     + ## Makefile ##
     +@@ Makefile: TEST_BUILTINS_OBJS += test-wildmatch.o
     + TEST_BUILTINS_OBJS += test-windows-named-pipe.o
     + TEST_BUILTINS_OBJS += test-write-cache.o
     + TEST_BUILTINS_OBJS += test-xml-encode.o
     ++TEST_BUILTINS_OBJS += test-hide-refs.o
     + 
     + # Do not add more tests here unless they have extra dependencies. Add
     + # them in TEST_BUILTINS_OBJS above.
     +
     + ## t/helper/test-hide-refs.c (new) ##
     +@@
     ++#include "cache.h"
     ++#include "hash.h"
     ++#include "config.h"
     ++#include "connect.h"
     ++#include "parse-options.h"
     ++#include "pkt-line.h"
     ++#include "sigchain.h"
     ++#include "test-tool.h"
     ++
     ++static const char *hide_refs_usage[] = {
     ++	"test-tool hide-refs [<options>...]",
     ++	NULL
     ++};
     ++
     ++static int die_before_read_ref;
     ++static int die_after_proc_ref;
     ++static int version = 1;
     ++static int hash_size = GIT_SHA1_HEXSZ;
     ++static struct string_list hidelist = STRING_LIST_INIT_NODUP;
     ++
     ++static void hide_refs_verison(struct packet_reader *reader) {
     ++	int server_version = 0;
     ++
     ++	for (;;) {
     ++		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
     ++			break;
     ++
     ++		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
     ++			server_version = atoi(reader->line+8);
     ++			if (server_version != 1)
     ++				die("bad protocol version: %d", server_version);
     ++		}
     ++	}
     ++
     ++	packet_write_fmt(1, "version=%d\n", version);
     ++	packet_flush(1);
     ++}
     ++
     ++static void hide_refs_proc(struct packet_reader *reader)
     ++{
     ++	const char *p;
     ++	struct strbuf buf = STRBUF_INIT;
     ++	enum packet_read_status status;
     ++
     ++	if (die_before_read_ref)
     ++		die("die with the --die-before-read-ref option");
     ++
     ++	for (;;) {
     ++		status = packet_reader_read(reader);
     ++		if (status == PACKET_READ_EOF)
     ++			exit(0);
     ++
     ++		if (status != PACKET_READ_NORMAL)
     ++			break;
     ++
     ++		p = reader->line;
     ++		strbuf_reset(&buf);
     ++		strbuf_addstr(&buf, reader->line);
     ++	}
     ++
     ++	p = strchr(buf.buf, ':');
     ++	if (unsorted_string_list_has_string(&hidelist, p + 1)) {
     ++		packet_write_fmt(1, "hide");
     ++	}
     ++
     ++	if (die_after_proc_ref)
     ++		die("die with the --die-after-proc-refs option");
     ++
     ++	packet_flush(1);
     ++}
     ++
     ++int cmd__hide_refs(int argc, const char **argv) {
     ++	int nongit_ok = 0;
     ++	struct packet_reader reader;
     ++	const char *value = NULL;
     ++	struct option options[] = {
     ++		OPT_BOOL(0, "die-before-read-ref", &die_before_read_ref,
     ++			 "die when reading first reference"),
     ++		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
     ++			 "die after proc ref"),
     ++		OPT_STRING_LIST('H', "hide", &hidelist, "refs-to-force-hidden",
     ++				"refs that will be force hidden"),
     ++		OPT_INTEGER('V', "version", &version,
     ++			    "use this protocol version number"),
     ++		OPT_END()
     ++	};
     ++
     ++	setup_git_directory_gently(&nongit_ok);
     ++
     ++	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
     ++	if (argc > 0)
     ++		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
     ++
     ++	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
     ++
     ++	if (!git_config_get_value("extensions.objectformat", &value)) {
     ++		if (!strcmp(value, "sha256"))
     ++			hash_size = GIT_SHA256_HEXSZ;
     ++	}
     ++
     ++	hide_refs_verison(&reader);
     ++	for (;;) {
     ++		hide_refs_proc(&reader);
     ++	}
     ++
     ++	return 0;
     ++}
     +
     + ## t/helper/test-tool.c ##
     +@@ t/helper/test-tool.c: static struct test_cmd cmds[] = {
     + 	{ "regex", cmd__regex },
     + 	{ "repository", cmd__repository },
     + 	{ "revision-walking", cmd__revision_walking },
     ++	{ "hide-refs", cmd__hide_refs },
     + 	{ "run-command", cmd__run_command },
     + 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
     + 	{ "serve-v2", cmd__serve_v2 },
     +
     + ## t/helper/test-tool.h ##
     +@@ t/helper/test-tool.h: int cmd__reftable(int argc, const char **argv);
     + int cmd__regex(int argc, const char **argv);
     + int cmd__repository(int argc, const char **argv);
     + int cmd__revision_walking(int argc, const char **argv);
     ++int cmd__hide_refs(int argc, const char **argv);
     + int cmd__run_command(int argc, const char **argv);
     + int cmd__scrap_cache_tree(int argc, const char **argv);
     + int cmd__serve_v2(int argc, const char **argv);
     +
       ## t/t1419-hide-refs-hook.sh (new) ##
      @@
      +#!/bin/sh
     @@ t/t1419-hide-refs-hook.sh (new)
      +
      +test_description='Test hide-refs hook'
      +
     -+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
     -+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
     -+
      +. ./test-lib.sh
     -+
      +. "$TEST_DIRECTORY"/t1419/common-functions.sh
      +
     -+setup_bare_repo_and_work_repo () {
     -+	# Refs of bare_repo : main(A)
     -+	# Refs of work_repo: main(A)  tags/v123
     ++GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
     ++export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
     ++
     ++setup_test_repos () {
      +	test_expect_success "setup bare_repo and work_repo" '
     -+		rm -rf bare_repo.git bare_repo.git.dump &&
     -+		rm -rf work_repo work_repo.dump &&
     ++		rm -rf bare_repo.git &&
     ++		rm -rf work_repo &&
      +		git init --bare bare_repo.git &&
      +		git init work_repo &&
     ++
     ++		# create new commits and references
      +		create_commits_in work_repo A B C D &&
      +		(
      +			cd work_repo &&
      +			git config --local core.abbrev 7 &&
     -+			git remote add origin ../bare_repo.git &&
     -+			git update-ref refs/heads/dev $A &&
     -+			git update-ref refs/heads/main $B &&
     ++			git update-ref refs/heads/main $A &&
     ++			git update-ref refs/heads/dev $B &&
      +			git update-ref refs/pull-requests/1/head $C &&
      +			git tag -m "v123" v123 $D &&
     -+			git push origin +refs/heads/*:refs/heads/* &&
     -+			git push origin +refs/tags/*:refs/tags/* &&
     -+			git push origin +refs/pull-requests/*:refs/pull-requests/*
     ++			git push ../bare_repo.git +refs/heads/*:refs/heads/* &&
     ++			git push ../bare_repo.git +refs/tags/*:refs/tags/* &&
     ++			git push ../bare_repo.git +refs/pull-requests/*:refs/pull-requests/*
      +		) &&
      +		TAG=$(git -C work_repo rev-parse v123) &&
      +
     -+		# setup pre-receive hook
     -+		write_script bare_repo.git/hooks/pre-receive <<-\EOF &&
     -+		exec >&2
     -+		echo "# pre-receive hook"
     -+		while read old new ref
     -+		do
     -+			echo "pre-receive< $old $new $ref"
     -+		done
     -+		EOF
     -+
     -+		# setup update hook
     -+		write_script bare_repo.git/hooks/update <<-\EOF &&
     -+		exec >&2
     -+		echo "# update hook"
     -+		echo "update< $@"
     -+		EOF
     -+
     -+		# setup post-receive hook
     -+		write_script bare_repo.git/hooks/post-receive <<-\EOF
     -+		exec >&2
     -+		echo "# post-receive hook"
     -+		while read old new ref
     -+		do
     -+			echo "post-receive< $old $new $ref"
     -+		done
     -+		EOF
     ++		# config transfer.hiderefs values with "hook:" prefix
     ++		(
     ++			git -C bare_repo.git config --local http.receivepack true &&
     ++			git -C bare_repo.git config --add transfer.hiderefs hook:HEAD &&
     ++			git -C bare_repo.git config --add transfer.hiderefs hook:refs
     ++		)
      +	'
      +}
      +
     -+run_hide_refs_hook_tests() {
     -+	case $1 in
     -+		http)
     -+			PROTOCOL="HTTP protocol"
     -+			BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
     -+			BAREREPO_PREFIX="$HTTPD_URL"/smart
     -+			;;
     -+		local)
     -+			PROTOCOL="builtin protocol"
     -+			BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
     -+			BAREREPO_PREFIX="$(pwd)"
     -+			;;
     -+	esac
     -+
     -+	BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
     -+
     -+	GIT_TEST_PROTOCOL_VERSION=$2
     -+
     -+	# Run test cases for 'hide-refs' hook
     -+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
     ++setup_httpd() {
     ++	ROOT_PATH="$PWD"
     ++	. "$TEST_DIRECTORY"/lib-gpg.sh
     ++	. "$TEST_DIRECTORY"/lib-httpd.sh
     ++	. "$TEST_DIRECTORY"/lib-terminal.sh
     ++
     ++	start_httpd
     ++	set_askpass user@host pass@host
     ++	setup_askpass_helper
     ++}
     ++
     ++# Run test cases when hide-refs hook exit abnormally
     ++run_tests_for_abnormal_hook() {
     ++	GIT_TEST_PROTOCOL_VERSION=$1
     ++	BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
     ++
     ++	for t in  "$TEST_DIRECTORY"/t1419/abnormal-*.sh
      +	do
     -+		# Initialize the bare_repo repository and work_repo
     -+		setup_bare_repo_and_work_repo
     -+		git -C work_repo remote set-url origin "$BAREREPO_URL"
     -+		cp -rf work_repo work_repo.dump
     -+
     -+		git -C bare_repo.git config --local http.receivepack true
     -+		git -C bare_repo.git config --add transfer.hiderefs force:HEAD
     -+		git -C bare_repo.git config --add transfer.hiderefs force:refs
     -+		cp -rf bare_repo.git bare_repo.git.dump
     -+
     -+		if test "$1" = "http"; then
     -+			setup_askpass_helper
     -+			rm -rf "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
     -+			mv bare_repo.git "$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
     -+		fi
     ++		setup_test_repos
      +
      +		. "$t"
      +	done
      +}
      +
     ++# Run test cases under local/HTTP protocol
     ++run_tests_for_normal_hook() {
     ++	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
     ++	do
     ++		setup_test_repos
     ++		case $1 in
     ++			http)
     ++				PROTOCOL="HTTP protocol"
     ++
     ++				# bare_repo.git need move to httpd sever root path
     ++				BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
     ++				rm -rf "$BAREREPO_GIT_DIR"
     ++				mv bare_repo.git "$BAREREPO_GIT_DIR"
     ++
     ++				# setup the repository service URL address of http protocol
     ++				BAREREPO_PREFIX="$HTTPD_URL"/smart
     ++				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
     ++				;;
     ++			local)
     ++				PROTOCOL="builtin protocol"
     ++				BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
     ++
     ++				# setup the repository service address of builtin protocol
     ++				BAREREPO_PREFIX="$(pwd)"
     ++				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
     ++				;;
     ++		esac
     ++
     ++		GIT_TEST_PROTOCOL_VERSION=$2
     ++		git -C work_repo remote add origin "$BAREREPO_URL"
      +
     -+setup_bare_repo_and_work_repo
     -+BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
     -+BAREREPO_PREFIX="$(pwd)"
     -+BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
     -+
     -+# Load test cases that only need to be executed once.
     -+for t in  "$TEST_DIRECTORY"/t1419/once-*.sh
     -+do
     -+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:HEAD
     -+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs force:refs
     -+	. "$t"
     -+done
     ++		. "$t"
     ++	done
     ++}
      +
     ++setup_httpd
      +for protocol in 1 2
      +do
     -+	# Run test cases for 'hide-refs' hook on local file protocol.
     -+	run_hide_refs_hook_tests local $protocol
     ++	run_tests_for_abnormal_hook $protocol
     ++	run_tests_for_normal_hook local $protocol
     ++	run_tests_for_normal_hook http $protocol
      +done
      +
     -+ROOT_PATH="$PWD"
     -+. "$TEST_DIRECTORY"/lib-gpg.sh
     -+. "$TEST_DIRECTORY"/lib-httpd.sh
     -+. "$TEST_DIRECTORY"/lib-terminal.sh
     ++test_done
     +
     + ## t/t1419/abnormal-hide-refs-hook.sh (new) ##
     +@@
     ++#!/bin/sh
      +
     -+start_httpd
     -+set_askpass user@host pass@host
     ++# Upstream repository (bare_repo.git) contains the configurations:
     ++#
     ++#	[transfer]
     ++#		hiderefs = hook:HEAD
     ++#		hiderefs = hook:refs
     ++#
     ++# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it,
     ++# we should make sure Git works correctly in some speicail cases
     ++
     ++# If hide-refs not exists, Git should not invoke it and continue advertise all the refs
     ++test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook not exists" '
     ++	rm -f "$BAREREPO_GIT_DIR/hooks/hide-refs" &&
     ++	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     ++	cat out | make_user_friendly_and_stable_output >actual &&
     ++	format_and_save_expect <<-EOF &&
     ++		<COMMIT-A> HEAD
     ++		<COMMIT-B> refs/heads/dev
     ++		<COMMIT-A> refs/heads/main
     ++		<COMMIT-C> refs/pull-requests/1/head
     ++		<COMMIT-TAG-v123> refs/tags/v123
     ++		<COMMIT-D> refs/tags/v123^{}
     ++	EOF
     ++	test_cmp expect actual
     ++'
      +
     -+# Run test cases for 'hide-refs' hook on HTTP protocol.
     -+for protocol in 1 2
     -+do
     -+	run_hide_refs_hook_tests http $protocol
     -+done
     ++# If hide-refs hook run with incompatible version, Git should not invoke it and continue to advertise all the refs
     ++test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook run with incompatible version" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
     ++		test-tool hide-refs --version=2
     ++	EOF
     ++	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     ++	cat out | make_user_friendly_and_stable_output >actual &&
     ++	format_and_save_expect <<-EOF &&
     ++		<COMMIT-A> HEAD
     ++		<COMMIT-B> refs/heads/dev
     ++		<COMMIT-A> refs/heads/main
     ++		<COMMIT-C> refs/pull-requests/1/head
     ++		<COMMIT-TAG-v123> refs/tags/v123
     ++		<COMMIT-D> refs/tags/v123^{}
     ++	EOF
     ++	test_cmp expect actual
     ++'
      +
     -+test_done
     ++# If hide-refs hook exit before processing any refs, Git should not die and continue to advertise all the refs
     ++test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die before read ref" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
     ++		test-tool hide-refs --die-before-read-ref
     ++	EOF
     ++	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     ++	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
     ++	format_and_save_expect <<-EOF &&
     ++		fatal: die with the --die-before-read-ref option
     ++		<COMMIT-A> HEAD
     ++		<COMMIT-B> refs/heads/dev
     ++		<COMMIT-A> refs/heads/main
     ++		<COMMIT-C> refs/pull-requests/1/head
     ++		<COMMIT-TAG-v123> refs/tags/v123
     ++		<COMMIT-D> refs/tags/v123^{}
     ++	EOF
     ++	test_cmp expect actual
     ++'
     ++
     ++# If hide-refs hook exit abnormally, Git should not die and continue to advertise left refs
     ++test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die after proc ref" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
     ++		test-tool hide-refs --die-after-proc-refs
     ++	EOF
     ++	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     ++	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
     ++	format_and_save_expect <<-EOF &&
     ++		fatal: die with the --die-after-proc-refs option
     ++		<COMMIT-A> HEAD
     ++		<COMMIT-B> refs/heads/dev
     ++		<COMMIT-A> refs/heads/main
     ++		<COMMIT-C> refs/pull-requests/1/head
     ++		<COMMIT-TAG-v123> refs/tags/v123
     ++		<COMMIT-D> refs/tags/v123^{}
     ++	EOF
     ++	test_cmp expect actual
     ++'
      
       ## t/t1419/common-functions.sh (new) ##
      @@
     @@ t/t1419/common-functions.sh (new)
      +	fi
      +}
      +
     -+# Format the output of git-push, git-show-ref and other commands to make a
     ++# Format the output of git-fetch, git-ls-remote and other commands to make a
      +# user-friendly and stable text.  We can easily prepare the expect text
      +# without having to worry about changes of the commit ID (full or abbrev.)
      +# of the output.  Single quotes are replaced with double quotes, because
     -+# it is boring to prepare unquoted single quotes in expect text.  We also
     -+# remove some locale error messages. The emitted human-readable errors are
     -+# redundant to the more machine-readable output the tests already assert.
     ++# it is boring to prepare unquoted single quotes in expect text.
      +make_user_friendly_and_stable_output () {
      +	tr '\0' '@' | sed \
      +		-e "s/'/\"/g" \
     @@ t/t1419/common-functions.sh (new)
      +		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
      +		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
      +		-e "s/$ZERO_OID/<ZERO-OID>/g" \
     -+		-e "s#$TRASH_DIRECTORY/bare_repo.git#<URL/of/bare_repo.git>#" \
     -+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#"
     ++		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#" \
     ++		-e 's/^[0-9a-f]\{4\}//g'
     ++
      +}
      +
      +filter_out_hide_refs_output() {
      +	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
      +}
      +
     -+filter_out_user_friendly_and_stable_output () {
     -+	make_user_friendly_and_stable_output |
     -+		sed -n ${1+"$@"}
     -+}
     -+
      +format_and_save_expect () {
      +	sed -e 's/^> //' -e 's/Z$//' >expect
      +}
     @@ t/t1419/common-functions.sh (new)
      +	test_cmp show-ref.expect show-ref.filtered
      +}
      
     - ## t/t1419/once-0000-abnormal-hide-refs-hook.sh (new) ##
     -@@
     -+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs \
     -+			--die-read-version \
     -+			-r refs/heads/main
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read version" '
     -+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-version option
     -+		fatal: can not read version message from hook hide-refs
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when write version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-write-version
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when write version" '
     -+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-write-version option
     -+		fatal: can not read version message from hook hide-refs
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read first filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-read-first-ref
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
     -+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-first-ref option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die when read second filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-read-second-ref
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
     -+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-second-ref option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): setup hide-refs hook which die while filtring refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-after-proc-refs
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 1): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
     -+	test_must_fail git -c protocol.version=1 upload-pack --advertise-refs "$BAREREPO_URL"  >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-after-proc-refs option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-read-version
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read version" '
     -+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-version option
     -+		fatal: can not read version message from hook hide-refs
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when write version" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-write-version
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when write version" '
     -+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-write-version option
     -+		fatal: can not read version message from hook hide-refs
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read first filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-read-first-ref
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read first filter request" '
     -+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-first-ref option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die when read second filter request" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-read-second-ref
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die when read second filter request" '
     -+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-read-second-ref option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): setup hide-refs hook which die while filtring refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs --die-after-proc-refs
     -+	EOF
     -+'
     -+
     -+test_expect_success "builtin protocol (protocol: 2): upload-pack --advertise-refs while hide-refs hook die while filtring refs" '
     -+	test_must_fail git -c protocol.version=2 upload-pack --advertise-refs "$BAREREPO_URL" >out 2>&1 &&
     -+	cat out | grep "fatal: " | make_user_friendly_and_stable_output >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		fatal: die with the --die-after-proc-refs option
     -+		fatal: hook hide-refs died abnormally
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -
       ## t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs
     -+	EOF
     -+'
     ++#!/bin/sh
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-B>	HEAD
     -+		<COMMIT-A>	refs/heads/dev
     -+		<COMMIT-B>	refs/heads/main
     -+		<COMMIT-C>	refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123>	refs/tags/v123
     -+		<COMMIT-D>	refs/tags/v123^{}
     -+	EOF
     -+	test_cmp expect actual
     -+'
     ++# Upstream repository (bare_repo.git) contains the configurations:
     ++#
     ++#	[transfer]
     ++#		hiderefs = hook:HEAD
     ++#		hiderefs = hook:refs
     ++#
     ++# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++# Git will not advertise the refs that are hided by hide-refs hook
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide part of refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev" \
     -+		-r "refs/heads/main" \
     -+		-r "refs/pull-requests/1/head" \
     -+		-r "refs/tags/v123"
     ++		-H "refs/pull-requests/1/head" \
     ++		-H "refs/tags/v123"
      +	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide all refs" '
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     ++		<COMMIT-A>	HEAD
     ++		<COMMIT-B>	refs/heads/dev
     ++		<COMMIT-A>	refs/heads/main
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev" \
     -+		-r "refs/heads/main"
     ++# The hide-ref hook should not change the default effects of [transfer|uploadpack|receive].hiderefs configurations,
     ++# if it hide no refs, the original hiderefs rules should works
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
     ++		test-tool hide-refs
      +	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide branches" '
     ++	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs refs/heads/dev &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
      +	format_and_save_expect <<-EOF &&
     ++		<COMMIT-A>	HEAD
     ++		<COMMIT-A>	refs/heads/main
      +		<COMMIT-C>	refs/pull-requests/1/head
      +		<COMMIT-TAG-v123>	refs/tags/v123
      +		<COMMIT-D>	refs/tags/v123^{}
      +	EOF
      +	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+	test-tool hide-refs \
     -+		-r "refs/pull-requests/1/head" \
     -+		-r "refs/tags/v123"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide pull refs and tags" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-B>	HEAD
     -+		<COMMIT-A>	refs/heads/dev
     -+		<COMMIT-B>	refs/heads/main
     -+	EOF
     -+	test_cmp expect actual
      +'
      
       ## t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+		test-tool hide-refs
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide no refs" '
     -+	rm -rf local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -C local.git show-ref -d >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     -+		<COMMIT-C> refs/pull-requests/1/head
     -+		<COMMIT-TAG-v123> refs/tags/v123
     -+		<COMMIT-D> refs/tags/v123^{}
     -+	EOF
     -+	test_cmp expect actual
     -+'
     ++#!/bin/sh
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev" \
     -+		-r "refs/heads/main" \
     -+		-r "refs/pull-requests/1/head" \
     -+		-r "refs/tags/v123"
     -+	EOF
     -+'
     ++# Upstream repository (bare_repo.git) contains the configurations:
     ++#
     ++#	[transfer]
     ++#		hiderefs = hook:HEAD
     ++#		hiderefs = hook:refs
     ++#
     ++# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide all refs" '
     ++# Git client can not fetch the refs that are hided by hide-refs hook
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide part of refs" '
      +	rm -rf local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	test_must_fail git -C local.git show-ref -d >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev" \
     -+		-r "refs/heads/main"
     ++		-H "HEAD" \
     ++		-H "refs/heads/dev" \
     ++		-H "refs/heads/main"
      +	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone branches" '
     -+	rm -rf local.git &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
      +	git -C local.git show-ref -d >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >actual &&
     @@ t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which some branches" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     -+	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a tip commit which is not hidden" '
     -+	rm -rf local.git &&
     -+	git init local.git &&
     -+	git -C local.git remote add origin "$BAREREPO_URL" &&
     -+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
     -+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
     -+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is not hidden" '
     ++# If a ref is hided by hide-refs hook, its private commits (tip or non-tip) will be force hidden
     ++# to the client, and the client can not fetch such kind of commit even the server set allowTipSHA1InWant
     ++# or allowReachableSHA1InWant to true
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a commit which is hided by hide-refs hook" '
      +	rm -rf local.git &&
     -+	git init local.git &&
     -+	git -C local.git remote add origin "$BAREREPO_URL" &&
     -+	git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $A
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide pull refs and tags" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +	test-tool hide-refs \
     -+		-r "refs/pull-requests/1/head" \
     -+		-r "refs/tags/v123"
     -+	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hook hide pull refs and tags" '
     -+	rm -rf local.git &&
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
     -+	git -C local.git show-ref -d >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		<COMMIT-A> refs/heads/dev
     -+		<COMMIT-B> refs/heads/main
     ++		-H "refs/heads/dev" \
     ++		-H "refs/pull-requests/1/head" \
     ++		-H "refs/tags/v123"
      +	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): config allowAnySHA1InWant to true" '
      +	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
     -+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a non-tip commit which is hidden" '
     -+	rm -rf local.git &&
     ++	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true &&
      +	git init local.git &&
      +	git -C local.git remote add origin "$BAREREPO_URL" &&
     -+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $C
     ++	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
      +'
      
       ## t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh (new) ##
      @@
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide no refs" '
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++#!/bin/sh
     ++
     ++# Upstream repository (bare_repo.git) contains the configurations:
     ++#
     ++#	[transfer]
     ++#		hiderefs = hook:HEAD
     ++#		hiderefs = hook:refs
     ++#
     ++# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
     ++
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook does not hide it" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +		test-tool hide-refs
      +	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide no refs" '
      +	create_commits_in work_repo E &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
      +	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/main <COMMIT-B> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-B> <COMMIT-E> refs/heads/main        Z
      +		To <URL/of/bare_repo.git>
     -+		   <COMMIT-B>..<COMMIT-E>  HEAD -> main
     ++		   <COMMIT-A>..<COMMIT-E>  HEAD -> main
      +	EOF
      +	test_cmp expect actual
      +'
      +
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): create ref while hide-refs hook hide no refs" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:new >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/new <ZERO-OID> <COMMIT-E>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <ZERO-OID> <COMMIT-E> refs/heads/new        Z
     -+		To <URL/of/bare_repo.git>
     -+		 * [new branch]      HEAD -> new
     -+		EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to delete ref while hide-refs hook hide no refs" '
     -+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin :dev >out 2>&1 &&
     -+	make_user_friendly_and_stable_output <out >out.tmp &&
     -+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
     -+	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		remote: # update hook        Z
     -+		remote: update< refs/heads/dev <COMMIT-A> <ZERO-OID>        Z
     -+		remote: # post-receive hook        Z
     -+		remote: post-receive< <COMMIT-A> <ZERO-OID> refs/heads/dev        Z
     -+		To <URL/of/bare_repo.git>
     -+		 - [deleted]         dev
     -+	EOF
     -+	test_cmp expect actual
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): setup hide-refs hook which hide all refs" '
     -+	rm -rf work_repo &&
     -+	cp -rf work_repo.dump work_repo &&
     -+	rm -rf "$BAREREPO_GIT_DIR" &&
     -+	cp -rf bare_repo.git.dump "$BAREREPO_GIT_DIR" &&
     -+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF
     ++# If hide-refs hook hide some ref, git push will be rejected
     ++test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide it" '
     ++	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +	test-tool hide-refs \
     -+		-r "HEAD" \
     -+		-r "refs/heads/dev" \
     -+		-r "refs/heads/main" \
     -+		-r "refs/pull-requests/1/head" \
     -+		-r "refs/tags/v123"
     ++		-H "refs/heads/main"
      +	EOF
     -+'
     -+
     -+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide all refs" '
     -+	create_commits_in work_repo E &&
     ++	create_commits_in work_repo F &&
      +	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
      +	make_user_friendly_and_stable_output <out >out.tmp &&
      +	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
      +	format_and_save_expect <<-EOF &&
     -+		remote: # pre-receive hook        Z
     -+		remote: pre-receive< <ZERO-OID> <COMMIT-E> refs/heads/main        Z
      +		To <URL/of/bare_repo.git>
      +		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
      +		error: failed to push some refs to "<URL/of/bare_repo.git>"
 3:  8c5ae78de47 ! 5:  8a5f7762c27 doc: add documentation for the hide-refs hook
     @@
       ## Metadata ##
     -Author: Sun Chao <16657101987@163.com>
     +Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
          doc: add documentation for the hide-refs hook
      
     -    "git upload-pack" or "git receive-pack" can use "hide-refs"
     -    hook to filter the references during reference discovery phase.
     +    If uploadpack.allowTipSHA1InWant or
     +    uploadpack.allowReachableSHA1InWant are set to true, the private
     +    commits of hide refs can be fetched by client. The new "hide-refs"
     +    hook are used to hide our refs and we wish to hide the private
     +    commits either.
     +
     +    "git upload-pack" or "git receive-pack" can use "hide-refs" hook to
     +    filter the references during reference discovery phase. If a ref is
     +    hided by "hide-refs" hook, its private data cannot be fetched by
     +    client even if uploadpack.allowTipSHA1InWant or
     +    uploadpack.allowReachableSHA1InWant are set to true.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +hook. In the following example, the letter 'G' stands for 'git-receive-pack'
      +or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
      +decides if the reference will be hidden or not, it sends result back in
     -+pkt-line format protocol, a response "hide" the references will hide
     -+to the client and can not fetch it even in protocol V2.
     ++pkt-line format protocol, a response "hide" the references will hidden
     ++to the client.
      +
      +	# Version negotiation
      +	G: PKT-LINE(version=1\0uploadpack)
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +	# Case 2: this reference can be advertised
      +	H: flush-pkt
      +
     -+To enable the `hide-refs` hook, we should config hiderefs with `force:`
     ++To enable the `hide-refs` hook, we should config hiderefs with `hook:`
      +option, eg:
      +
     -+	git config --add transfer.hiderefs force:refs/prefix1/
     -+	git config --add uploadpack.hiderefs force:!refs/prefix2/
     ++	git config --add transfer.hiderefs hook:refs/prefix1/
     ++	git config --add uploadpack.hiderefs hook:!refs/prefix2/
      +
      +the `hide-refs` will be called during reference discovery phase and
      +check each matched reference, a 'hide' response means the reference will

-- 
gitgitgadget

[PATCH v5 1/5] hiderefs: add hide-refs hook to hide refs dynamically

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Gerrit is implemented by JGit and is known as a centralized workflow system
which supports reference-level access control for repository. If we choose
to work in centralized workflow like what Gerrit provided, reference-level
access control is needed and we might add a reference filter hook
`hide-refs` to hide the private data.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will be
hidden to the client.

        # Version negotiation
        G: PKT-LINE(version=1\0uploadpack)
        G: flush-pkt
        H: PKT-LINE(version=1)
        H: flush-pkt

        # Send reference filter request to hook
        G: PKT-LINE(ref <refname>:<refname_full>)
        G: flush-pkt

        # Receive result from the hook.
        # Case 1: this reference is hidden
        H: PKT-LINE(hide)
        H: flush-pkt

        # Case 2: this reference can be advertised
        H: flush-pkt

To enable the `hide-refs` hook, we should config hiderefs with `hook:`
option, eg:

        git config --add transfer.hiderefs hook:refs/prefix1/
        git config --add uploadpack.hiderefs hook:!refs/prefix2/

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 refs.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 219 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index 92819732ab7..a99734fedcd 100644
--- a/refs.c
+++ b/refs.c
@@ -8,6 +8,7 @@
 #include "lockfile.h"
 #include "iterator.h"
 #include "refs.h"
+#include "pkt-line.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
 #include "hook.h"
@@ -1384,10 +1385,14 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 }
 
 static struct string_list *hide_refs;
+static struct string_list *hook_hide_refs;
+static struct strbuf hide_refs_section = STRBUF_INIT;
 
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
 	const char *key;
+	int hook = 0;
+
 	if (!strcmp("transfer.hiderefs", var) ||
 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
 	     !strcmp(key, "hiderefs"))) {
@@ -1396,27 +1401,218 @@ int parse_hide_refs_config(const char *var, const char *value, const char *secti
 
 		if (!value)
 			return config_error_nonbool(var);
+
+		/*
+		 * the prefix 'hook:' means that the matched refs will be
+		 * checked by the hide-refs hook dynamically, we need to put
+		 * the 'ref' string to the hook_hide_refs list
+		 */
+		if (skip_prefix(value, "hook:", &value)) {
+			if (!strlen(value))
+				return error(_("missing value for '%s' after hook option"), var);
+			hook = 1;
+		}
+
 		ref = xstrdup(value);
 		len = strlen(ref);
 		while (len && ref[len - 1] == '/')
 			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
+
+		if (hook) {
+			if (!hook_hide_refs) {
+				CALLOC_ARRAY(hook_hide_refs, 1);
+				hook_hide_refs->strdup_strings = 1;
+			}
+			string_list_append(hook_hide_refs, ref);
+		} else {
+			if (!hide_refs) {
+				CALLOC_ARRAY(hide_refs, 1);
+				hide_refs->strdup_strings = 1;
+			}
+			string_list_append(hide_refs, ref);
 		}
-		string_list_append(hide_refs, ref);
 	}
+
+	/*
+	 * Once hide-refs hook is invoked, Git need to do version negotiation,
+	 * with it, version number and process name ('uploadpack' or 'receive')
+	 * will send to it in pkt-line format, the proccess name is recorded
+	 * by hide_refs_section
+	 */
+	if (hook && hide_refs_section.len == 0)
+		strbuf_addstr(&hide_refs_section, section);
+
 	return 0;
 }
 
-int ref_is_hidden(const char *refname, const char *refname_full)
+static struct child_process *hide_refs_proc;
+static struct packet_reader *hide_refs_reader;
+
+/*
+ * Create the hide-refs hook child process and complete version negotiation,
+ * return non-zero upon success, otherwise 0
+ */
+static int create_hide_refs_process(void)
+{
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int version = 0;
+	int err;
+
+	hook_path = find_hook("hide-refs");
+	if (!hook_path)
+		return 0;
+
+	proc = (struct child_process *)xcalloc(1, sizeof (struct child_process));
+	reader = (struct packet_reader *)xcalloc(1, sizeof(struct packet_reader));
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "hide-refs";
+	proc->err = 0;
+
+	err = start_command(proc);
+	if (err)
+		goto cleanup;
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+	err = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
+	if (!err)
+		err = packet_flush_gently(proc->in);
+
+	if (!err)
+		for (;;) {
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether hide-refs exited abnormally */
+				if (status == PACKET_READ_EOF)
+					goto failure;
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+			}
+		}
+
+	if (err)
+		goto failure;
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		trace_printf(_("hook hide-refs version '%d' is not supported"), version);
+		goto failure;
+	}
+
+	sigchain_pop(SIGPIPE);
+
+	hide_refs_proc = proc;
+	hide_refs_reader = reader;
+	return 1;
+
+failure:
+	close(proc->in);
+	close(proc->out);
+	kill(proc->pid, SIGTERM);
+	finish_command_in_signal(proc);
+
+cleanup:
+	free(proc);
+	free(reader);
+	sigchain_pop(SIGPIPE);
+	return 0;
+}
+
+/* If hide-refs child process start failed, set skip_hide_refs_proc to true */
+static int skip_hide_refs_proc;
+
+/*
+ * Return non-zero if hide-refs hook want to hide the ref and 0 otherwise,
+ * and return 0 if hide-refs child proccess start failed or exit abnormally
+ */
+static int ref_hidden_check_by_hook(const char *refname, const char *refname_full)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int err;
+	int ret = 0;
+
+	if (skip_hide_refs_proc)
+		return 0;
+
+	if (!hide_refs_proc)
+		if (!create_hide_refs_process()) {
+			skip_hide_refs_proc = 1;
+			return 0;
+		}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	err = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
+	if (err)
+		goto cleanup;
+
+	err = packet_flush_gently(hide_refs_proc->in);
+	if (err)
+		goto cleanup;
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(hide_refs_reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether hide-refs exited abnormally */
+			if (status == PACKET_READ_EOF)
+				goto cleanup;
+			break;
+		}
+
+		strbuf_addstr(&buf, hide_refs_reader->line);
+	}
+
+	if (!strncmp("hide", buf.buf, 4))
+		ret = 1;
+
+	sigchain_pop(SIGPIPE);
+	return ret;
+
+cleanup:
+	close(hide_refs_proc->in);
+	close(hide_refs_proc->out);
+	kill(hide_refs_proc->pid, SIGTERM);
+	finish_command_in_signal(hide_refs_proc);
+
+	free(hide_refs_proc);
+	free(hide_refs_reader);
+	sigchain_pop(SIGPIPE);
+
+	skip_hide_refs_proc = 1;
+	return 0;
+}
+
+static int ref_hidden_check(const char *refname, const char *refname_full, int hook)
 {
+	struct string_list *hide_refs_list = hide_refs;
 	int i;
 
-	if (!hide_refs)
+	if (hook)
+		hide_refs_list = hook_hide_refs;
+
+	if (!hide_refs_list)
 		return 0;
-	for (i = hide_refs->nr - 1; i >= 0; i--) {
-		const char *match = hide_refs->items[i].string;
+	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
+		const char *match = hide_refs_list->items[i].string;
 		const char *subject;
 		int neg = 0;
 		const char *p;
@@ -1436,12 +1632,25 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 		/* refname can be NULL when namespaces are used. */
 		if (subject &&
 		    skip_prefix(subject, match, &p) &&
-		    (!*p || *p == '/'))
-			return !neg;
+		    (!*p || *p == '/')) {
+			if (neg)
+				return 0;
+			if (!hook)
+				return 1;
+			return ref_hidden_check_by_hook(refname, refname_full);
+		}
 	}
 	return 0;
 }
 
+int ref_is_hidden(const char *refname, const char *refname_full)
+{
+	if (ref_hidden_check(refname, refname_full, 0) ||
+	    ref_hidden_check(refname, refname_full, 1))
+		return 1;
+	return 0;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
-- 
gitgitgadget

Re: [PATCH v5 1/5] hiderefs: add hide-refs hook to hide refs dynamically

[Junio C Hamano]

"Sun Chao via GitGitGadget" <gitgitgadget@gmail.com> writes:

> From: Sun Chao <sunchao9@huawei.com>
>
> Gerrit is implemented by JGit and is known as a centralized workflow system
> which supports reference-level access control for repository. If we choose
> to work in centralized workflow like what Gerrit provided, reference-level
> access control is needed and we might add a reference filter hook
> `hide-refs` to hide the private data.

Please rewrite the above so that it does not sound like "Gerrit
supports it, there are tons of users of Gerrit, we must support it,
too".  If this feature is meaningful for us, even if Gerrit folks
were deprecating and planning to remove the support of it, we would
add it.  If it is not, even if Gerrit folks support it, we wouldn't.

> +
> +		/*
> +		 * the prefix 'hook:' means that the matched refs will be
> +		 * checked by the hide-refs hook dynamically, we need to put
> +		 * the 'ref' string to the hook_hide_refs list
> +		 */

I am not sure if this deserves a five-line comment.  We didn't need
to have a comment that says "value without hook: means the matched
refs will be hidden and we need to remember them in the hide_refs
string_list" for over 10 years after all.

> +		if (skip_prefix(value, "hook:", &value)) {
> +			if (!strlen(value))
> +				return error(_("missing value for '%s' after hook option"), var);

I am not sure it is a good idea to special case an empty string,
especially here at this point in the code flow.  There would be
strings that cannot be a refname prefix (e.g. "foo..bar") and such a
check is better done at the place where the accumuldated list of ref
patterns are actually used.  If you are using prefix match, a value
of an empty string here would be a very natural way to say "we pass
all the refs through our hook".

By the way, how does the negated entry work with this new one?  For
static ones,

	[transfer] hiderefs = !refs/heads/

would hide everything other than refs/heads/ hierarchy, I suppose.
Would we spell

	[transfer] hiderefs = hook:!refs/heads/

or

	[transfer] hiderefs = !hook:refs/heads/

to say "send everything outside the branches to hook"?  If the
former, you'd also need to special case "!" the same way as you
special case an empty string (in short, I am saying that the special
case only for an empty string does not make much sense).

How does this mechanism work with gitnamespaces (see "git config --help"
and read on transfer.hideRerfs)?

> +			hook = 1;
> +		}
> +
>  		ref = xstrdup(value);
>  		len = strlen(ref);
>  		while (len && ref[len - 1] == '/')
>  			ref[--len] = '\0';
> -		if (!hide_refs) {
> -			CALLOC_ARRAY(hide_refs, 1);
> -			hide_refs->strdup_strings = 1;
> +
> +		if (hook) {
> +			if (!hook_hide_refs) {
> +				CALLOC_ARRAY(hook_hide_refs, 1);
> +				hook_hide_refs->strdup_strings = 1;
> +			}
> +			string_list_append(hook_hide_refs, ref);
> +		} else {
> +			if (!hide_refs) {
> +				CALLOC_ARRAY(hide_refs, 1);
> +				hide_refs->strdup_strings = 1;
> +			}
> +			string_list_append(hide_refs, ref);
>  		}
> -		string_list_append(hide_refs, ref);
>  	}

That's a somewhat duplicated code.  I wonder

	/* no need for "hook" variable anymore */
	struct string_list **refs_list= &hide_refs;

	if (strip "hook:" prefix from value)
		refs_list = &hook_hide_refs;
		...
	if (!*refs_list) {
        	*refs_list = xcalloc(1, sizeof(*refs_list));
		(*refs_list)->strdup_strings = 1;
	}
	string_list_append(*refs_list, ref);
		
would be a better organization.  I dunno.

> +
> +	/*
> +	 * Once hide-refs hook is invoked, Git need to do version negotiation,
> +	 * with it, version number and process name ('uploadpack' or 'receive')
> +	 * will send to it in pkt-line format, the proccess name is recorded
> +	 * by hide_refs_section
> +	 */

Grammar.

> +	if (hook && hide_refs_section.len == 0)
> +		strbuf_addstr(&hide_refs_section, section);
> +

I am not sure if this is correct at all, but because the 1/N patch
has only code without documentation I cannot guess the intention.

The first conditional to parse the configuration variable name var
tries to handle both generic transfer.hideRefs and direction
specific {receive,uploadpack}.hideRefs and "section" at this point
has "transfer", "receive" or "uploadpack", doesn't it?

As this is a git_config() callback, when we have

	[receive] hiderefs = hook:refs/foo
	[uploadpack] hiderefs = hook:refs/bar
	[transfer] hiderefs = hook:refs/baz

we would want to send refs/bar and refs/baz to the hook if we are a
"uploadpack" process.  But because the above code records the first
section we happen to see (which is "receive"), hide_refs_section has
that value.  I am not sure how a code that later user that piece of
information can behave any sensibly.  Does it say "We are a
'uploadpack', but hide_refs_section says 'receive', so we should
ignore what is in hook_hide_refs string list"?

I'll stop reading at this point for now, as it is not a good use of
our time to review the implementation until we know the basic design
is sound, which I do not quite see from what we saw up to this
point.  It might have made sense if each string list element had the
ref pattern to match as its value and stored extra info, like "is
this negated?", "is this hook pattern or static?", "is this
transfer, receive, or uploadpack?" in its .util member, for example.

Thanks.

Re: [PATCH v5 1/5] hiderefs: add hide-refs hook to hide refs dynamically

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> The first conditional to parse the configuration variable name var
> tries to handle both generic transfer.hideRefs and direction
> specific {receive,uploadpack}.hideRefs and "section" at this point
> has "transfer", "receive" or "uploadpack", doesn't it?

This part of my review comments was based on my misreading the
patch.  Sorry.  "section" comes from the caller and says either
"receive" or "uploadpack".

But because the updating of hide_refs_section is done outside that
"first conditional" that makes sure we are actually looking at a
"*.hiderefs" configuration variable, it is misleading.  The only
saving grace is that it is guarded by "hook"

> +	if (hook && hide_refs_section.len == 0)
> +		strbuf_addstr(&hide_refs_section, section);
> +

that is only set inside the body of the if statement of the first
conditional that ensures that we are reading *.hiderefs variable,
but it would make more sense to move it inside it.

Or even better would be to clean the function up with a preliminary
patch to return early when we are not looking at *.hiderefs variable,
perhaps like the attached, and then build on top.

 refs.c | 39 ++++++++++++++++++++++-----------------
 1 file changed, 22 insertions(+), 17 deletions(-)

diff --git i/refs.c w/refs.c
index c89d558892..8cf8c58ebd 100644
--- i/refs.c
+++ w/refs.c
@@ -1393,24 +1393,29 @@ static struct string_list *hide_refs;
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
 	const char *key;
-	if (!strcmp("transfer.hiderefs", var) ||
-	    (!parse_config_key(var, section, NULL, NULL, &key) &&
-	     !strcmp(key, "hiderefs"))) {
-		char *ref;
-		int len;
-
-		if (!value)
-			return config_error_nonbool(var);
-		ref = xstrdup(value);
-		len = strlen(ref);
-		while (len && ref[len - 1] == '/')
-			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
-		}
-		string_list_append(hide_refs, ref);
+	char *ref;
+	int len;
+
+	/*
+	 * "section" is either "receive" or "uploadpack"; are we looking
+	 * at transfer.hiderefs or $section.hiderefs?
+	 */
+	if (strcmp("transfer.hiderefs", var) &&
+	    !(!parse_config_key(var, section, NULL, NULL, &key) &&
+	      !strcmp(key, "hiderefs")))
+		return 0; /* neither */
+	if (!value)
+		return config_error_nonbool(var);
+	ref = xstrdup(value);
+	len = strlen(ref);
+	while (len && ref[len - 1] == '/')
+		ref[--len] = '\0';
+	if (!hide_refs) {
+		CALLOC_ARRAY(hide_refs, 1);
+		hide_refs->strdup_strings = 1;
 	}
+	string_list_append(hide_refs, ref);
+
 	return 0;
 }
 

Re: [PATCH v5 1/5] hiderefs: add hide-refs hook to hide refs dynamically

[孙超]

> On Sep 14, 2022, at 01:01, Junio C Hamano <gitster@pobox.com> wrote:
> 
>> Gerrit is implemented by JGit and is known as a centralized workflow system
>> which supports reference-level access control for repository. If we choose
>> to work in centralized workflow like what Gerrit provided, reference-level
>> access control is needed and we might add a reference filter hook
>> `hide-refs` to hide the private data.
> 
> Please rewrite the above so that it does not sound like "Gerrit
> supports it, there are tons of users of Gerrit, we must support it,
> too".  If this feature is meaningful for us, even if Gerrit folks
> were deprecating and planning to remove the support of it, we would
> add it.  If it is not, even if Gerrit folks support it, we wouldn't.

Hi Junio, thanks for your advice here, I cannot agree with you more, I will do it.

> 
>> +
>> +		/*
>> +		 * the prefix 'hook:' means that the matched refs will be
>> +		 * checked by the hide-refs hook dynamically, we need to put
>> +		 * the 'ref' string to the hook_hide_refs list
>> +		 */
> 
> I am not sure if this deserves a five-line comment.  We didn't need
> to have a comment that says "value without hook: means the matched
> refs will be hidden and we need to remember them in the hide_refs
> string_list" for over 10 years after all.

Agree, and I will remove them.

> 
>> +		if (skip_prefix(value, "hook:", &value)) {
>> +			if (!strlen(value))
>> +				return error(_("missing value for '%s' after hook option"), var);
> 
> I am not sure it is a good idea to special case an empty string,
> especially here at this point in the code flow.  There would be
> strings that cannot be a refname prefix (e.g. "foo..bar") and such a
> check is better done at the place where the accumuldated list of ref
> patterns are actually used.  If you are using prefix match, a value
> of an empty string here would be a very natural way to say "we pass
> all the refs through our hook".

Yes, this is a good advice. Previously I cannot pass all the refs through
the new hook unless set two config items like:

         [transfer]
             hiderefs = hook:HEAD
             hiderefs = hook:refs

I thinks it is a good idea to use only one config item to replace them:

         [transfer] hiderefs = hook:
> 
> By the way, how does the negated entry work with this new one?  For
> static ones,
> 
> 	[transfer] hiderefs = !refs/heads/
> 
> would hide everything other than refs/heads/ hierarchy, I suppose.
> Would we spell
> 
> 	[transfer] hiderefs = hook:!refs/heads/
> 
> or
> 
> 	[transfer] hiderefs = !hook:refs/heads/
> 
> to say "send everything outside the branches to hook"?  If the
> former, you'd also need to special case "!" the same way as you
> special case an empty string (in short, I am saying that the special
> case only for an empty string does not make much sense).

In my patch I put the "!" after the "hook:", and negate passing all the refs to the
hook would like

         [transfer] hiderefs = hook:!

however according to the match mechanism of hiderefs, it will be better to delete
the config item above. If there are no config item, the hook will not be called.

So if I want to pass all the refs but some scope of them, it will be like (use a empty
string to match all the refs)

         [transfer]
             hiderefs = hook:
             hiderefs = hook:!refs/pull/

which means pass all the refs except for the ones begins with 'refs/pull/'


> How does this mechanism work with gitnamespaces (see "git config --help"
> and read on transfer.hideRerfs)?

In my patch Git will send refname and refnamefull(with namepsace) to the hook, the hook
will check it and response with 'hide' or not. In the following example, the letter
'G' stands for 'git-receive-pack' or 'git-upload-pack' and the letter 'H' stands for
this hook

       # Send reference filter request to hook
       G: PKT-LINE(ref <refname>:<refnamefull>)
       G: flush-pkt

       # Receive result from the hook.
       # Case 1: this reference is hidden
       H: PKT-LINE(hide)
       H: flush-pkt

       # Case 2: this reference can be advertised
       H: flush-pkt

I'm not sure if it is suitable or not, I think it will be better to send both the refname
and the refnamefull to the hook.

> That's a somewhat duplicated code.  I wonder
> 
> 	/* no need for "hook" variable anymore */
> 	struct string_list **refs_list= &hide_refs;
> 
> 	if (strip "hook:" prefix from value)
> 		refs_list = &hook_hide_refs;
> 		...
> 	if (!*refs_list) {
>        	*refs_list = xcalloc(1, sizeof(*refs_list));
> 		(*refs_list)->strdup_strings = 1;
> 	}
> 	string_list_append(*refs_list, ref);
> 		
> would be a better organization.  I dunno.

Agree, it looks better, I will do it.

> 
>> +
>> +	/*
>> +	 * Once hide-refs hook is invoked, Git need to do version negotiation,
>> +	 * with it, version number and process name ('uploadpack' or 'receive')
>> +	 * will send to it in pkt-line format, the proccess name is recorded
>> +	 * by hide_refs_section
>> +	 */
> 
> Grammar.

Will fix.

> On Sep 17, 2022, at 01:52, Junio C Hamano <gitster@pobox.com> wrote:
> 
> Junio C Hamano <gitster@pobox.com> writes:
> 
> ... ...
> 
>> +	if (hook && hide_refs_section.len == 0)
>> +		strbuf_addstr(&hide_refs_section, section);
>> +
> 
> that is only set inside the body of the if statement of the first
> conditional that ensures that we are reading *.hiderefs variable,
> but it would make more sense to move it inside it.

Agree, it will be better to move it inside.

> 
> Or even better would be to clean the function up with a preliminary
> patch to return early when we are not looking at *.hiderefs variable,
> perhaps like the attached, and then build on top.
> 
> ... ...
> 
> +
> +	/*
> +	 * "section" is either "receive" or "uploadpack"; are we looking
> +	 * at transfer.hiderefs or $section.hiderefs?
> +	 */
> +	if (strcmp("transfer.hiderefs", var) &&
> +	    !(!parse_config_key(var, section, NULL, NULL, &key) &&
> +	      !strcmp(key, "hiderefs")))
> +		return 0; /* neither */
> +	if (!value)
> +		return config_error_nonbool(var);
> +	ref = xstrdup(value);
> +	len = strlen(ref);
> +	while (len && ref[len - 1] == '/')
> +		ref[--len] = '\0';
> +	if (!hide_refs) {
> +		CALLOC_ARRAY(hide_refs, 1);
> +		hide_refs->strdup_strings = 1;
> 	}
> +	string_list_append(hide_refs, ref);
> +
> 	return 0;
> }

Thanks for the advice here, I Will do it.

[PATCH v5 2/5] hiderefs: use new flag to mark force hidden refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

If uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
are set to true, the private commits of hide refs can be fetched by
client. The new hide-refs hook are used to hide our refs and we wish to
hide the private commits either.

Now we have hide-refs hook to hide refs dynamically, a new
flag `HIDDEN_REF_FORCE` is used to mark a ref if hide-refs hook
decide to hide it, and we make sure the wire protocol V1 will reject
to send the private commits of these refs even if
uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
are set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 refs.c        | 39 +++++++++++++++++++++++++++++++++++++++
 refs.h        |  3 +++
 upload-pack.c | 14 --------------
 3 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/refs.c b/refs.c
index a99734fedcd..3fbdf967bc6 100644
--- a/refs.c
+++ b/refs.c
@@ -1651,6 +1651,45 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 	return 0;
 }
 
+#define OUR_REF		(1u << 12)
+#define HIDDEN_REF	(1u << 19)
+/*
+ * Use this flag to mark a ref that is hided by hide-refs hook, its private
+ * commits (tip or non-tip commits, not reachable by the refs not hided by
+ * hide-refs hook) will be force hidden to the client, which means client can
+ * not fetch such kind of commit even uploadpack.allowTipSHA1InWant or
+ * uploadpack.allowReachableSHA1InWant are set to true
+ */
+#define HIDDEN_REF_FORCE	(1u << 20)
+
+static unsigned int ref_hidden_flag(const char *refname, const char *refname_full)
+{
+	if (ref_hidden_check(refname, refname_full, 1))
+		return HIDDEN_REF_FORCE;
+	else if (ref_hidden_check(refname, refname_full, 0))
+		return HIDDEN_REF;
+	return OUR_REF;
+}
+
+int mark_our_ref(const char *refname, const char *refname_full,
+		 const struct object_id *oid)
+{
+	struct object *o;
+	unsigned int flag;
+
+	if (!oid || is_null_oid(oid)) {
+		return 0;
+	}
+
+	o = lookup_unknown_object(the_repository, oid);
+	flag = ref_hidden_flag(refname, refname_full);
+	o->flags |= flag;
+
+	if (flag & OUR_REF)
+		return 0;
+	return 1;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
diff --git a/refs.h b/refs.h
index d6575b8c2bd..2feabfe35c4 100644
--- a/refs.h
+++ b/refs.h
@@ -819,6 +819,9 @@ int parse_hide_refs_config(const char *var, const char *value, const char *);
  * parameter always points to the full ref name.
  */
 int ref_is_hidden(const char *, const char *);
+/* return non-zero if the ref is hidden, otherwise 0 */
+int mark_our_ref(const char *refname, const char *refname_full,
+		 const struct object_id *oid);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/upload-pack.c b/upload-pack.c
index b217a1f469e..a8ca5d1c26e 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1155,20 +1155,6 @@ static void receive_needs(struct upload_pack_data *data,
 		packet_flush(1);
 }
 
-/* return non-zero if the ref is hidden, otherwise 0 */
-static int mark_our_ref(const char *refname, const char *refname_full,
-			const struct object_id *oid)
-{
-	struct object *o = lookup_unknown_object(the_repository, oid);
-
-	if (ref_is_hidden(refname, refname_full)) {
-		o->flags |= HIDDEN_REF;
-		return 1;
-	}
-	o->flags |= OUR_REF;
-	return 0;
-}
-
 static int check_ref(const char *refname_full, const struct object_id *oid,
 		     int flag, void *cb_data)
 {
-- 
gitgitgadget

[PATCH v5 3/5] hiderefs: hornor hide flags in wire protocol V2

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Previously hide refs can not protect private data of hide refs in wire
protocol V2, for example a `ALL_FLAGS` will be used to clear all the
objects before handling the fetch requests.

Hornor the hide flags by removing `HIDDEN_REFS` flag from `ALL_FLAGS`
and make sure all the refs will check its hidden flags before sending pack
to client, especially during stateless rpc. And if there are refs with
`HIDDEN_REF_FORCE` flag, use `check_non_tip` to protect the private date
of force hidden refs.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 ls-refs.c     |  2 +-
 refs.c        | 20 ++++++++++++++++++++
 refs.h        |  4 ++++
 upload-pack.c | 11 +++++++++--
 4 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..b5cb1316d38 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 	strbuf_reset(&data->buf);
 
-	if (ref_is_hidden(refname_nons, refname))
+	if (mark_our_ref(refname_nons, refname, oid))
 		return 0;
 
 	if (!ref_match(&data->prefixes, refname_nons))
diff --git a/refs.c b/refs.c
index 3fbdf967bc6..1424de0048e 100644
--- a/refs.c
+++ b/refs.c
@@ -1662,6 +1662,25 @@ int ref_is_hidden(const char *refname, const char *refname_full)
  */
 #define HIDDEN_REF_FORCE	(1u << 20)
 
+/* Use this variable to record existing object hidden flags */
+static unsigned int objects_hidden_flags;
+
+/* Return non-zero if need to batch check hidden refs, otherwise 0 */
+int need_check_hidden_refs(void)
+{
+	if (!objects_hidden_flags)
+		return 1;
+	return 0;
+}
+
+/* Return non-zero if some ref is force hidden, otherwise 0 */
+int has_force_hidden_refs(void)
+{
+	if (objects_hidden_flags & HIDDEN_REF_FORCE)
+		return 1;
+	return 0;
+}
+
 static unsigned int ref_hidden_flag(const char *refname, const char *refname_full)
 {
 	if (ref_hidden_check(refname, refname_full, 1))
@@ -1684,6 +1703,7 @@ int mark_our_ref(const char *refname, const char *refname_full,
 	o = lookup_unknown_object(the_repository, oid);
 	flag = ref_hidden_flag(refname, refname_full);
 	o->flags |= flag;
+	objects_hidden_flags |= flag;
 
 	if (flag & OUR_REF)
 		return 0;
diff --git a/refs.h b/refs.h
index 2feabfe35c4..8deb36a95cc 100644
--- a/refs.h
+++ b/refs.h
@@ -822,6 +822,10 @@ int ref_is_hidden(const char *, const char *);
 /* return non-zero if the ref is hidden, otherwise 0 */
 int mark_our_ref(const char *refname, const char *refname_full,
 		 const struct object_id *oid);
+/* return non-zero if need to batch check hidden refs, otherwise 0 */
+int need_check_hidden_refs(void);
+/* return non-zero if some ref is force hidden, otherwise 0 */
+int has_force_hidden_refs(void);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/upload-pack.c b/upload-pack.c
index a8ca5d1c26e..a9a24399d8e 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -39,8 +39,8 @@
 #define CLIENT_SHALLOW	(1u << 18)
 #define HIDDEN_REF	(1u << 19)
 
-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
+		NOT_SHALLOW | CLIENT_SHALLOW)
 
 /* Enum for allowed unadvertised object request (UOR) */
 enum allow_uor {
@@ -1726,6 +1726,13 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 				state = FETCH_DONE;
 			break;
 		case FETCH_SEND_PACK:
+			if (need_check_hidden_refs()) {
+				head_ref_namespaced(check_ref, NULL);
+				for_each_namespaced_ref(check_ref, NULL);
+			}
+			if (has_force_hidden_refs())
+				check_non_tip(&data);
+
 			send_wanted_ref_info(&data);
 			send_shallow_info(&data);
 
-- 
gitgitgadget

[PATCH v5 4/5] test: add test cases for hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'hide-refs' hook which is used to
filter the references during reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                                      |   1 +
 t/helper/test-hide-refs.c                     | 107 +++++++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 112 ++++++++++++++++++
 t/t1419/abnormal-hide-refs-hook.sh            |  82 +++++++++++++
 t/t1419/common-functions.sh                   |  74 ++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  45 +++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh |  47 ++++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  42 +++++++
 10 files changed, 512 insertions(+)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh

diff --git a/Makefile b/Makefile
index 924b864ae83..c6793681b41 100644
--- a/Makefile
+++ b/Makefile
@@ -794,6 +794,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-hide-refs.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
diff --git a/t/helper/test-hide-refs.c b/t/helper/test-hide-refs.c
new file mode 100644
index 00000000000..751fc6213f3
--- /dev/null
+++ b/t/helper/test-hide-refs.c
@@ -0,0 +1,107 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *hide_refs_usage[] = {
+	"test-tool hide-refs [<options>...]",
+	NULL
+};
+
+static int die_before_read_ref;
+static int die_after_proc_ref;
+static int version = 1;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list hidelist = STRING_LIST_INIT_NODUP;
+
+static void hide_refs_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	for (;;) {
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+		}
+	}
+
+	packet_write_fmt(1, "version=%d\n", version);
+	packet_flush(1);
+}
+
+static void hide_refs_proc(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+
+	if (die_before_read_ref)
+		die("die with the --die-before-read-ref option");
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = strchr(buf.buf, ':');
+	if (unsorted_string_list_has_string(&hidelist, p + 1)) {
+		packet_write_fmt(1, "hide");
+	}
+
+	if (die_after_proc_ref)
+		die("die with the --die-after-proc-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__hide_refs(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "die-before-read-ref", &die_before_read_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
+			 "die after proc ref"),
+		OPT_STRING_LIST('H', "hide", &hidelist, "refs-to-force-hidden",
+				"refs that will be force hidden"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	hide_refs_verison(&reader);
+	for (;;) {
+		hide_refs_proc(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 80055886798..c5bd7f1f806 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -72,6 +72,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "hide-refs", cmd__hide_refs },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index a432cc77d92..cba8d3b093d 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -61,6 +61,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__hide_refs(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/t/t1419-hide-refs-hook.sh b/t/t1419-hide-refs-hook.sh
new file mode 100755
index 00000000000..3b505f0fa9d
--- /dev/null
+++ b/t/t1419-hide-refs-hook.sh
@@ -0,0 +1,112 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test hide-refs hook'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+setup_test_repos () {
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git &&
+		rm -rf work_repo &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+
+		# create new commits and references
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git update-ref refs/heads/main $A &&
+			git update-ref refs/heads/dev $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push ../bare_repo.git +refs/heads/*:refs/heads/* &&
+			git push ../bare_repo.git +refs/tags/*:refs/tags/* &&
+			git push ../bare_repo.git +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# config transfer.hiderefs values with "hook:" prefix
+		(
+			git -C bare_repo.git config --local http.receivepack true &&
+			git -C bare_repo.git config --add transfer.hiderefs hook:HEAD &&
+			git -C bare_repo.git config --add transfer.hiderefs hook:refs
+		)
+	'
+}
+
+setup_httpd() {
+	ROOT_PATH="$PWD"
+	. "$TEST_DIRECTORY"/lib-gpg.sh
+	. "$TEST_DIRECTORY"/lib-httpd.sh
+	. "$TEST_DIRECTORY"/lib-terminal.sh
+
+	start_httpd
+	set_askpass user@host pass@host
+	setup_askpass_helper
+}
+
+# Run test cases when hide-refs hook exit abnormally
+run_tests_for_abnormal_hook() {
+	GIT_TEST_PROTOCOL_VERSION=$1
+	BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+
+	for t in  "$TEST_DIRECTORY"/t1419/abnormal-*.sh
+	do
+		setup_test_repos
+
+		. "$t"
+	done
+}
+
+# Run test cases under local/HTTP protocol
+run_tests_for_normal_hook() {
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		setup_test_repos
+		case $1 in
+			http)
+				PROTOCOL="HTTP protocol"
+
+				# bare_repo.git need move to httpd sever root path
+				BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+				rm -rf "$BAREREPO_GIT_DIR"
+				mv bare_repo.git "$BAREREPO_GIT_DIR"
+
+				# setup the repository service URL address of http protocol
+				BAREREPO_PREFIX="$HTTPD_URL"/smart
+				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+				;;
+			local)
+				PROTOCOL="builtin protocol"
+				BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+
+				# setup the repository service address of builtin protocol
+				BAREREPO_PREFIX="$(pwd)"
+				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+				;;
+		esac
+
+		GIT_TEST_PROTOCOL_VERSION=$2
+		git -C work_repo remote add origin "$BAREREPO_URL"
+
+		. "$t"
+	done
+}
+
+setup_httpd
+for protocol in 1 2
+do
+	run_tests_for_abnormal_hook $protocol
+	run_tests_for_normal_hook local $protocol
+	run_tests_for_normal_hook http $protocol
+done
+
+test_done
diff --git a/t/t1419/abnormal-hide-refs-hook.sh b/t/t1419/abnormal-hide-refs-hook.sh
new file mode 100644
index 00000000000..c615a74f3a0
--- /dev/null
+++ b/t/t1419/abnormal-hide-refs-hook.sh
@@ -0,0 +1,82 @@
+#!/bin/sh
+
+# Upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer]
+#		hiderefs = hook:HEAD
+#		hiderefs = hook:refs
+#
+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it,
+# we should make sure Git works correctly in some speicail cases
+
+# If hide-refs not exists, Git should not invoke it and continue advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook not exists" '
+	rm -f "$BAREREPO_GIT_DIR/hooks/hide-refs" &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If hide-refs hook run with incompatible version, Git should not invoke it and continue to advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook run with incompatible version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --version=2
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If hide-refs hook exit before processing any refs, Git should not die and continue to advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die before read ref" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --die-before-read-ref
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-before-read-ref option
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If hide-refs hook exit abnormally, Git should not die and continue to advertise left refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die after proc ref" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..7841241b038
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,74 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-fetch, git-ls-remote and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e 's/^[0-9a-f]\{4\}//g'
+
+}
+
+filter_out_hide_refs_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..9803994b6a8
--- /dev/null
+++ b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+
+# Upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer]
+#		hiderefs = hook:HEAD
+#		hiderefs = hook:refs
+#
+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
+
+# Git will not advertise the refs that are hided by hide-refs hook
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide part of refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/pull-requests/1/head" \
+		-H "refs/tags/v123"
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A>	HEAD
+		<COMMIT-B>	refs/heads/dev
+		<COMMIT-A>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+# The hide-ref hook should not change the default effects of [transfer|uploadpack|receive].hiderefs configurations,
+# if it hide no refs, the original hiderefs rules should works
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs
+	EOF
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs refs/heads/dev &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A>	HEAD
+		<COMMIT-A>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..d1174266fd3
--- /dev/null
+++ b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
@@ -0,0 +1,47 @@
+#!/bin/sh
+
+# Upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer]
+#		hiderefs = hook:HEAD
+#		hiderefs = hook:refs
+#
+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
+
+# Git client can not fetch the refs that are hided by hide-refs hook
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide part of refs" '
+	rm -rf local.git &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "HEAD" \
+		-H "refs/heads/dev" \
+		-H "refs/heads/main"
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If a ref is hided by hide-refs hook, its private commits (tip or non-tip) will be force hidden
+# to the client, and the client can not fetch such kind of commit even the server set allowTipSHA1InWant
+# or allowReachableSHA1InWant to true
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a commit which is hided by hide-refs hook" '
+	rm -rf local.git &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/heads/dev" \
+		-H "refs/pull-requests/1/head" \
+		-H "refs/tags/v123"
+	EOF
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
+'
diff --git a/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..0fe3c2e1389
--- /dev/null
+++ b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
@@ -0,0 +1,42 @@
+#!/bin/sh
+
+# Upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer]
+#		hiderefs = hook:HEAD
+#		hiderefs = hook:refs
+#
+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook does not hide it" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs
+	EOF
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		To <URL/of/bare_repo.git>
+		   <COMMIT-A>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+# If hide-refs hook hide some ref, git push will be rejected
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide it" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/heads/main"
+	EOF
+	create_commits_in work_repo F &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH v5 5/5] doc: add documentation for the hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

If uploadpack.allowTipSHA1InWant or
uploadpack.allowReachableSHA1InWant are set to true, the private
commits of hide refs can be fetched by client. The new "hide-refs"
hook are used to hide our refs and we wish to hide the private
commits either.

"git upload-pack" or "git receive-pack" can use "hide-refs" hook to
filter the references during reference discovery phase. If a ref is
hided by "hide-refs" hook, its private data cannot be fetched by
client even if uploadpack.allowTipSHA1InWant or
uploadpack.allowReachableSHA1InWant are set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 48 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 48 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..314bddedc1f 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,54 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[hide-refs]]
+hide-refs
+~~~~~~~~~
+
+This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
+during the reference discovery phase, each reference and will be filtered
+by this hook. The hook executes once with no arguments for each
+'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
+a version number and server process name ('uploadpack' or 'receive') will
+send to it in pkt-line format, followed by a flush-pkt. The hook should
+respond with its version number.
+
+During reference discovery phase, each reference will be filtered by this
+hook. In the following example, the letter 'G' stands for 'git-receive-pack'
+or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
+decides if the reference will be hidden or not, it sends result back in
+pkt-line format protocol, a response "hide" the references will hidden
+to the client.
+
+	# Version negotiation
+	G: PKT-LINE(version=1\0uploadpack)
+	G: flush-pkt
+	H: PKT-LINE(version=1)
+	H: flush-pkt
+
+	# Send reference filter request to hook
+	G: PKT-LINE(ref <refname>:<refnamefull>)
+	G: flush-pkt
+
+	# Receive result from the hook.
+	# Case 1: this reference is hidden
+	H: PKT-LINE(hide)
+	H: flush-pkt
+
+	# Case 2: this reference can be advertised
+	H: flush-pkt
+
+To enable the `hide-refs` hook, we should config hiderefs with `hook:`
+option, eg:
+
+	git config --add transfer.hiderefs hook:refs/prefix1/
+	git config --add uploadpack.hiderefs hook:!refs/prefix2/
+
+the `hide-refs` will be called during reference discovery phase and
+check each matched reference, a 'hide' response means the reference will
+be hidden for its private data even if `allowTipSHA1InWant` and
+`allowReachableSHA1InWant` are set to true.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget

[PATCH v6 0/5] hiderefs: add hide-refs hook to hide refs dynamically

[Sun Chao via GitGitGadget]

Hi, thanks for reviewing the patches, we'd like to add a new hook called
"hide-refs", which can be used to hide refs dynamically according to the
user's access to the git servers. Here is the reasons:

(1) There are lots of development teams in our company (Huawei), and they
have been using the Gerrit platform during development work for a long time
and get used to its features, e.g. Gerrit's reference-level control feature
can be used to arrange different small teams to develop on different
branches, some branches will be only visible to special teams, this way of
working effectively shields the interference between different branches.

In this way Gerrit could protect each teams' codes privately until
administrators decided to merge or cherry-pick the codes together. Most of
the developers cannot see all the references of the repositories, and Gerrit
does not apply the 'fork' feature which exists in Github or Gitlab, if you
want to create a code review task you can just run git command like:

git push origin HEAD:refs/for/main


Developers cannot fork their own repositories but clone from and push to the
same repositories, so in Huawei, when we talk about Gerrit, we always treat
it as a centralized workflow platform and lots of teams like its features.

(2) Gerrit implement the JGit which supports the wire git protocol V1 and
V2. We have built our new git server based on Gitlab-CE
[https://gitlab.com/rluna-gitlab/gitlab-ce] and it uses CGit (Compared to
Gerrit's JGit, someone call the Git itself as CGit). Recently we were
planning to migrate some development teams from Gerrit platform to the new
server. So we wish the new server can work like Gerrit which allow users
create code review task by git push command and also can hide refs according
to the user's access.

Commit 15d3af5e22 (receive-pack: add new proc-receive hook, 2020-08-27) make
it possible to create a code review by a single proc-receive command on the
server side. What we need right now is reference-level permission control,
we referred to Gerrit's implementation mechanism and added a new reference
filtering mechanism:

a) We config the reference level control rules on server side, eg:

    ```json
    {
        {
            "ref":"refs/heads/stable/*",
            "action":"read",
            "access":"allow",
            "user_group":"dev_group_1"
        },
        {
            "ref":"refs/heads/stable/secret_feature",
            "action":"read",
            "access":"deny",
            "user_group":"dev_group_1"
        },
        {
            "ref":"refs/for/*",
            "action":"create-codereview",
            "access":"access",
            "user_group":"dev_group_1"
        },
        ... ...
    }
    ```

b) During upload-pack and receive-pack the hide-refs hook will hide the refs
    that the user cannot read during reference advertise phase according to
    the reference level control rules above.
c) We also add a `HIDDEN_REF_FORCE` flag to make sure the private data
    of these hide refs cannot be fetched by the git clients.


(3) When we talk about permission control in git, it may comes to the
directory permission control, but however we can use the workflow like
Gerrit to control the contents of a repository by put its parts to different
branches, and merge or cherry-pick them together if needed.

In special scenarios, we need to protect the content on some branches from
being seen by others and leaked to other teams, now we can achieve that by
the new "hide-refs" hook.

And here I will talk about how the "hide-refs" works on server side:

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered with
this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, it sends result back in
pkt-line format protocol, a response "hide" means the references will hide
to the client.

# Version negotiation
G: PKT-LINE(version=1\0uploadpack)
G: flush-pkt
H: PKT-LINE(version=1)
H: flush-pkt

# Send reference filter request to hook
G: PKT-LINE(ref <refname>:<refname_full>)
G: flush-pkt

# Receive result from the hook.
# Case 1: this reference is hidden
H: PKT-LINE(hide)
H: flush-pkt

# Case 2: this reference can be advertised
H: flush-pkt


To enable the hide-refs hook, we should config hiderefs with hook: option,
e.g. if we want to pass all the refs to the new hook except for the tags:

git config --add transfer.hiderefs hook:
git config --add transfer.hiderefs hook:!refs/tags/


the hide-refs will be called during reference discovery phase and check each
matched reference, a 'hide' response means the reference will be hidden for
its private data even if allowTipSHA1InWant or allowReachableSHA1InWant are
set to true.

Sun Chao (5):
  hiderefs: add hide-refs hook to hide refs dynamically
  hiderefs: use a new flag to mark force hidden refs
  hiderefs: hornor hide flags in wire protocol V2
  test: add test cases for hide-refs hook
  doc: add documentation for the hide-refs hook

 Documentation/githooks.txt                    |  49 +++
 Makefile                                      |   1 +
 ls-refs.c                                     |   2 +-
 refs.c                                        | 309 ++++++++++++++++--
 refs.h                                        |   7 +
 t/helper/test-hide-refs.c                     | 107 ++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 111 +++++++
 t/t1419/abnormal-hide-refs-hook.sh            |  80 +++++
 t/t1419/common-functions.sh                   |  74 +++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  43 +++
 ...st-0002-upload-pack-with-hide-refs-hook.sh |  45 +++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  40 +++
 upload-pack.c                                 |  25 +-
 15 files changed, 856 insertions(+), 39 deletions(-)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh


base-commit: 79f2338b3746d23454308648b2491e5beba4beff
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1301%2Fsunchao9%2Frefs_advertise-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1301/sunchao9/refs_advertise-v6
Pull-Request: https://github.com/git/git/pull/1301

Range-diff vs v5:

 1:  278bd185aec ! 1:  99402c1b89f hiderefs: add hide-refs hook to hide refs dynamically
     @@ Metadata
       ## Commit message ##
          hiderefs: add hide-refs hook to hide refs dynamically
      
     -    Gerrit is implemented by JGit and is known as a centralized workflow system
     -    which supports reference-level access control for repository. If we choose
     -    to work in centralized workflow like what Gerrit provided, reference-level
     -    access control is needed and we might add a reference filter hook
     -    `hide-refs` to hide the private data.
     +    Muti-branch workflows are used in some development scenarios, especially
     +    for large teams, where different small teams are assigned to implement
     +    different features on different branches or to develop secret features
     +    on special branches.
     +
     +    If we can control the visible reference list based on developer
     +    permissions, we can reduce the interference between reference lists of
     +    different teams, and can achieve the protection of critical core code
     +    (only certain team members can see it before making it public). This
     +    kind of reference management makes sense, and on some platforms, sunch
     +    as Gerrit implement it through server-side reference access control.
     +
     +    We can use '{transfer,uploadpack,receive}.hiderefs' config items to
     +    control which references need to hide from clients, but the config items
     +    are static and cannot satisfy the above requirements. We need the Git
     +    server to hide references according to the user's permissions, we can
     +    try to implement this mechanism by introducing a server-side hook
     +    'hide-refs' to dynamically hide references during reference discovery
     +    phase.
      
          This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
          during the reference discovery phase, each reference will be filtered
     @@ Commit message
      
          During reference discovery phase, each reference will be filtered by this
          hook. In the following example, the letter 'G' stands for 'git-receive-pack'
     -    or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
     -    decides if the reference will be hidden or not, it sends result back in
     -    pkt-line format protocol, a response "hide" means the references will be
     -    hidden to the client.
     +    or 'git-upload-pack', and the letter 'H' stands for this hook. The hook
     +    decides if the reference will be hidden or not, and it sends the result
     +    back in pkt-line format protocol, a response "hide" means the references
     +    will be hidden to the client.
      
                  # Version negotiation
                  G: PKT-LINE(version=1\0uploadpack)
     @@ Commit message
                  H: flush-pkt
      
          To enable the `hide-refs` hook, we should config hiderefs with `hook:`
     -    option, eg:
     +    option, e.g. if we want to pass all the refs to the new hook except for
     +    the tags:
      
     -            git config --add transfer.hiderefs hook:refs/prefix1/
     -            git config --add uploadpack.hiderefs hook:!refs/prefix2/
     +            git config --add transfer.hiderefs hook:
     +            git config --add transfer.hiderefs hook:!refs/tags/
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ refs.c: char *shorten_unambiguous_ref(const char *refname, int strict)
       
       int parse_hide_refs_config(const char *var, const char *value, const char *section)
       {
     ++	struct string_list **refs_list= &hide_refs;
       	const char *key;
     -+	int hook = 0;
     +-	if (!strcmp("transfer.hiderefs", var) ||
     +-	    (!parse_config_key(var, section, NULL, NULL, &key) &&
     +-	     !strcmp(key, "hiderefs"))) {
     +-		char *ref;
     +-		int len;
     +-
     +-		if (!value)
     +-			return config_error_nonbool(var);
     +-		ref = xstrdup(value);
     +-		len = strlen(ref);
     +-		while (len && ref[len - 1] == '/')
     +-			ref[--len] = '\0';
     +-		if (!hide_refs) {
     +-			CALLOC_ARRAY(hide_refs, 1);
     +-			hide_refs->strdup_strings = 1;
     ++	char *ref;
     ++	int len;
      +
     - 	if (!strcmp("transfer.hiderefs", var) ||
     - 	    (!parse_config_key(var, section, NULL, NULL, &key) &&
     - 	     !strcmp(key, "hiderefs"))) {
     -@@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const char *secti
     - 
     - 		if (!value)
     - 			return config_error_nonbool(var);
     ++	if (strcmp("transfer.hiderefs", var) &&
     ++	    !(!parse_config_key(var, section, NULL, NULL, &key) &&
     ++	      !strcmp(key, "hiderefs")))
     ++		return 0;
     ++
     ++	if (!value)
     ++		return config_error_nonbool(var);
     ++
     ++	if (skip_prefix(value, "hook:", &value)) {
     ++		refs_list = &hook_hide_refs;
      +
      +		/*
     -+		 * the prefix 'hook:' means that the matched refs will be
     -+		 * checked by the hide-refs hook dynamically, we need to put
     -+		 * the 'ref' string to the hook_hide_refs list
     ++		 * Once the 'hide-refs' hook is invoked, Git needs to do
     ++		 * version negotiation with it, the version number and the
     ++		 * process name ('uploadpack' or 'receive') will send to
     ++		 * it in pkt-line format, and the process name is recorded
     ++		 * by hide_refs_section
      +		 */
     -+		if (skip_prefix(value, "hook:", &value)) {
     -+			if (!strlen(value))
     -+				return error(_("missing value for '%s' after hook option"), var);
     -+			hook = 1;
     -+		}
     ++		if (hide_refs_section.len == 0)
     ++			strbuf_addstr(&hide_refs_section, section);
     ++	}
      +
     - 		ref = xstrdup(value);
     - 		len = strlen(ref);
     - 		while (len && ref[len - 1] == '/')
     - 			ref[--len] = '\0';
     --		if (!hide_refs) {
     --			CALLOC_ARRAY(hide_refs, 1);
     --			hide_refs->strdup_strings = 1;
     ++	ref = xstrdup(value);
     ++	len = strlen(ref);
     ++	while (len && ref[len - 1] == '/')
     ++		ref[--len] = '\0';
      +
     -+		if (hook) {
     -+			if (!hook_hide_refs) {
     -+				CALLOC_ARRAY(hook_hide_refs, 1);
     -+				hook_hide_refs->strdup_strings = 1;
     -+			}
     -+			string_list_append(hook_hide_refs, ref);
     -+		} else {
     -+			if (!hide_refs) {
     -+				CALLOC_ARRAY(hide_refs, 1);
     -+				hide_refs->strdup_strings = 1;
     -+			}
     -+			string_list_append(hide_refs, ref);
     - 		}
     --		string_list_append(hide_refs, ref);
     - 	}
     ++	if (!*refs_list) {
     ++		CALLOC_ARRAY(*refs_list, 1);
     ++		(*refs_list)->strdup_strings = 1;
     ++	}
     ++	string_list_append(*refs_list, ref);
      +
     -+	/*
     -+	 * Once hide-refs hook is invoked, Git need to do version negotiation,
     -+	 * with it, version number and process name ('uploadpack' or 'receive')
     -+	 * will send to it in pkt-line format, the proccess name is recorded
     -+	 * by hide_refs_section
     -+	 */
     -+	if (hook && hide_refs_section.len == 0)
     -+		strbuf_addstr(&hide_refs_section, section);
     ++	return 0;
     ++}
      +
     - 	return 0;
     - }
     - 
     --int ref_is_hidden(const char *refname, const char *refname_full)
      +static struct child_process *hide_refs_proc;
      +static struct packet_reader *hide_refs_reader;
      +
     @@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const cha
      +			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
      +				version = atoi(reader->line + 8);
      +			}
     -+		}
     + 		}
     +-		string_list_append(hide_refs, ref);
      +
      +	if (err)
      +		goto failure;
     @@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const cha
      +	default:
      +		trace_printf(_("hook hide-refs version '%d' is not supported"), version);
      +		goto failure;
     -+	}
     + 	}
      +
      +	sigchain_pop(SIGPIPE);
      +
     @@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const cha
      +	free(proc);
      +	free(reader);
      +	sigchain_pop(SIGPIPE);
     -+	return 0;
     -+}
     -+
     + 	return 0;
     + }
     + 
     +-int ref_is_hidden(const char *refname, const char *refname_full)
      +/* If hide-refs child process start failed, set skip_hide_refs_proc to true */
      +static int skip_hide_refs_proc;
      +
     @@ refs.c: int parse_hide_refs_config(const char *var, const char *value, const cha
       {
      +	struct string_list *hide_refs_list = hide_refs;
       	int i;
     ++	int match_all = 0;
       
      -	if (!hide_refs)
      +	if (hook)
      +		hide_refs_list = hook_hide_refs;
     -+
      +	if (!hide_refs_list)
       		return 0;
      -	for (i = hide_refs->nr - 1; i >= 0; i--) {
      -		const char *match = hide_refs->items[i].string;
     ++
      +	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
      +		const char *match = hide_refs_list->items[i].string;
       		const char *subject;
       		int neg = 0;
       		const char *p;
     +@@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
     + 			match++;
     + 		}
     + 
     ++		/* empty string with the 'hook:' option matches all the refs */
     ++		if (hook && !*match) {
     ++			match_all = !neg;
     ++			continue;
     ++		}
     ++
     + 		if (*match == '^') {
     + 			subject = refname_full;
     + 			match++;
      @@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
       		/* refname can be NULL when namespaces are used. */
       		if (subject &&
     @@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
      +			return ref_hidden_check_by_hook(refname, refname_full);
      +		}
       	}
     - 	return 0;
     - }
     - 
     ++
     ++	if (hook && match_all)
     ++		return ref_hidden_check_by_hook(refname, refname_full);
     ++
     ++	return 0;
     ++}
     ++
      +int ref_is_hidden(const char *refname, const char *refname_full)
      +{
      +	if (ref_hidden_check(refname, refname_full, 0) ||
      +	    ref_hidden_check(refname, refname_full, 1))
      +		return 1;
     -+	return 0;
     -+}
     -+
     - const char *find_descendant_ref(const char *dirname,
     - 				const struct string_list *extras,
     - 				const struct string_list *skip)
     + 	return 0;
     + }
     + 
 2:  0df5ecc216d ! 2:  f309e9534f2 hiderefs: use new flag to mark force hidden refs
     @@ Metadata
      Author: Sun Chao <sunchao9@huawei.com>
      
       ## Commit message ##
     -    hiderefs: use new flag to mark force hidden refs
     +    hiderefs: use a new flag to mark force hidden refs
      
          If uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
     -    are set to true, the private commits of hide refs can be fetched by
     -    client. The new hide-refs hook are used to hide our refs and we wish to
     -    hide the private commits either.
     +    is set to true, the private commits of hiding refs can be fetched by
     +    the client. The new hide-refs hook is used to hide our refs and we wish
     +    to hide the private commits either.
      
     -    Now we have hide-refs hook to hide refs dynamically, a new
     -    flag `HIDDEN_REF_FORCE` is used to mark a ref if hide-refs hook
     -    decide to hide it, and we make sure the wire protocol V1 will reject
     -    to send the private commits of these refs even if
     +    A new flag `HIDDEN_REF_FORCE` is used to mark a ref if hide-refs hook
     +    decides to hide it, and we make sure the wire protocol V1 will reject
     +    to send the private commits of this kind of refs even if
          uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
     -    are set to true.
     +    is set to true.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ refs.c: int ref_is_hidden(const char *refname, const char *refname_full)
      +#define OUR_REF		(1u << 12)
      +#define HIDDEN_REF	(1u << 19)
      +/*
     -+ * Use this flag to mark a ref that is hided by hide-refs hook, its private
     -+ * commits (tip or non-tip commits, not reachable by the refs not hided by
     -+ * hide-refs hook) will be force hidden to the client, which means client can
     -+ * not fetch such kind of commit even uploadpack.allowTipSHA1InWant or
     ++ * Use this flag to mark a ref that is hidden by the hide-refs hook, its private
     ++ * commits (tip or non-tip commits, not reachable by the refs not hidden by the
     ++ * hide-refs hook) will be forced hidden to the client, which means a client can
     ++ * not fetch such kind of commits even uploadpack.allowTipSHA1InWant or
      + * uploadpack.allowReachableSHA1InWant are set to true
      + */
      +#define HIDDEN_REF_FORCE	(1u << 20)
 3:  de73f5a6fd9 ! 3:  0013476266e hiderefs: hornor hide flags in wire protocol V2
     @@ Metadata
       ## Commit message ##
          hiderefs: hornor hide flags in wire protocol V2
      
     -    Previously hide refs can not protect private data of hide refs in wire
     -    protocol V2, for example a `ALL_FLAGS` will be used to clear all the
     -    objects before handling the fetch requests.
     +    Previously hiderefs configurations can not protect the private data
     +    of hiding refs in wire protocol V2, for example, an `ALL_FLAGS` flag
     +    will be used to clear all the objects before handling the fetch requests.
      
     -    Hornor the hide flags by removing `HIDDEN_REFS` flag from `ALL_FLAGS`
     -    and make sure all the refs will check its hidden flags before sending pack
     -    to client, especially during stateless rpc. And if there are refs with
     -    `HIDDEN_REF_FORCE` flag, use `check_non_tip` to protect the private date
     -    of force hidden refs.
     +    Hornor the hide flags by removing the `HIDDEN_REFS` flag from the
     +    `ALL_FLAGS` and make sure all the refs will check its hidden flags
     +    before sending the pack to client, especially during stateless RPC.
     +    And if there are refs with `HIDDEN_REF_FORCE` flag, use `check_non_tip`
     +    to protect the private data of force-hidden refs.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
 4:  fb135cb3bd3 ! 4:  c1274d6b834 test: add test cases for hide-refs hook
     @@ Commit message
          test: add test cases for hide-refs hook
      
          Add test cases for the new 'hide-refs' hook which is used to
     -    filter the references during reference discovery phase.
     +    filter the references during the reference discovery phase.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ t/t1419-hide-refs-hook.sh (new)
      +		# config transfer.hiderefs values with "hook:" prefix
      +		(
      +			git -C bare_repo.git config --local http.receivepack true &&
     -+			git -C bare_repo.git config --add transfer.hiderefs hook:HEAD &&
     -+			git -C bare_repo.git config --add transfer.hiderefs hook:refs
     ++			git -C bare_repo.git config --add transfer.hiderefs hook:
      +		)
      +	'
      +}
     @@ t/t1419/abnormal-hide-refs-hook.sh (new)
      @@
      +#!/bin/sh
      +
     -+# Upstream repository (bare_repo.git) contains the configurations:
     ++# The upstream repository (bare_repo.git) contains the configurations:
      +#
     -+#	[transfer]
     -+#		hiderefs = hook:HEAD
     -+#		hiderefs = hook:refs
     ++#	[transfer] hiderefs = hook:
      +#
     -+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it,
     -+# we should make sure Git works correctly in some speicail cases
     ++# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it,
     ++# we should make sure Git works correctly in some special cases
      +
     -+# If hide-refs not exists, Git should not invoke it and continue advertise all the refs
     ++# If the hide-refs does not exist, Git should not invoke it and continue to advertise all the refs
      +test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook not exists" '
      +	rm -f "$BAREREPO_GIT_DIR/hooks/hide-refs" &&
      +	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
     @@ t/t1419/abnormal-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+# If hide-refs hook run with incompatible version, Git should not invoke it and continue to advertise all the refs
     ++# If the hide-refs hook run with incompatible version, Git should not invoke it and continue to advertise all the refs
      +test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook run with incompatible version" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +		test-tool hide-refs --version=2
     @@ t/t1419/abnormal-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+# If hide-refs hook exit before processing any refs, Git should not die and continue to advertise all the refs
     ++# If the hide-refs hook exit before processing any refs, Git should not die and continue to advertise all the refs
      +test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die before read ref" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +		test-tool hide-refs --die-before-read-ref
     @@ t/t1419/abnormal-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+# If hide-refs hook exit abnormally, Git should not die and continue to advertise left refs
     ++# If the hide-refs hook exit abnormally, Git should not die and continue to advertise left refs
      +test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die after proc ref" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +		test-tool hide-refs --die-after-proc-refs
     @@ t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh (new)
      @@
      +#!/bin/sh
      +
     -+# Upstream repository (bare_repo.git) contains the configurations:
     ++# The upstream repository (bare_repo.git) contains the configurations:
      +#
     -+#	[transfer]
     -+#		hiderefs = hook:HEAD
     -+#		hiderefs = hook:refs
     ++#	[transfer] hiderefs = hook:
      +#
     -+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
     ++# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
      +
     -+# Git will not advertise the refs that are hided by hide-refs hook
     ++# Git will not advertise the refs that are hidden by the hide-refs hook
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide part of refs" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +	test-tool hide-refs \
     @@ t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+# The hide-ref hook should not change the default effects of [transfer|uploadpack|receive].hiderefs configurations,
     -+# if it hide no refs, the original hiderefs rules should works
     ++# The hide-ref hook should not change the default effects of '{transfer,uploadpack,receive}.hiderefs'
     ++# configurations, if it hides no refs, the original hiderefs configurations should work
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
      +		test-tool hide-refs
     @@ t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh (new)
      @@
      +#!/bin/sh
      +
     -+# Upstream repository (bare_repo.git) contains the configurations:
     ++# The upstream repository (bare_repo.git) contains the configurations:
      +#
     -+#	[transfer]
     -+#		hiderefs = hook:HEAD
     -+#		hiderefs = hook:refs
     ++#	[transfer] hiderefs = hook:
      +#
     -+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
     ++# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
      +
     -+# Git client can not fetch the refs that are hided by hide-refs hook
     ++# Git client can not fetch the refs that are hidden by the hide-refs hook
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide part of refs" '
      +	rm -rf local.git &&
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
     @@ t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh (new)
      +	test_cmp expect actual
      +'
      +
     -+# If a ref is hided by hide-refs hook, its private commits (tip or non-tip) will be force hidden
     -+# to the client, and the client can not fetch such kind of commit even the server set allowTipSHA1InWant
     ++# If a ref is hidden by the hide-refs hook, its private commits (tip or non-tip) will be forced hidden
     ++# to the client, and the client can not fetch such kind of commits even if the server set allowTipSHA1InWant
      +# or allowReachableSHA1InWant to true
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a commit which is hided by hide-refs hook" '
      +	rm -rf local.git &&
     @@ t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh (new)
      @@
      +#!/bin/sh
      +
     -+# Upstream repository (bare_repo.git) contains the configurations:
     ++# The upstream repository (bare_repo.git) contains the configurations:
      +#
     -+#	[transfer]
     -+#		hiderefs = hook:HEAD
     -+#		hiderefs = hook:refs
     ++#	[transfer] hiderefs = hook:
      +#
     -+# During refs advertise phase the hide-refs hook will be invoked and each ref will be checked by it
     ++# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
      +
      +test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook does not hide it" '
      +	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
 5:  8a5f7762c27 ! 5:  0cb019b28da doc: add documentation for the hide-refs hook
     @@ Metadata
       ## Commit message ##
          doc: add documentation for the hide-refs hook
      
     -    If uploadpack.allowTipSHA1InWant or
     -    uploadpack.allowReachableSHA1InWant are set to true, the private
     -    commits of hide refs can be fetched by client. The new "hide-refs"
     -    hook are used to hide our refs and we wish to hide the private
     -    commits either.
     +    If uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
     +    is set to true, the private commits of hiding refs can be fetched by
     +    the client. The new "hide-refs" hook is used to hide our refs and we
     +    wish to hide the private commits either.
      
     -    "git upload-pack" or "git receive-pack" can use "hide-refs" hook to
     -    filter the references during reference discovery phase. If a ref is
     -    hided by "hide-refs" hook, its private data cannot be fetched by
     -    client even if uploadpack.allowTipSHA1InWant or
     -    uploadpack.allowReachableSHA1InWant are set to true.
     +    "git-upload-pack" or "git-receive-pack" can use "hide-refs" hook to
     +    filter the references during the reference discovery phase. If a ref
     +    is hidden by the "hide-refs" hook, its private data cannot be fetched
     +    by the client even if uploadpack.allowTipSHA1InWant or
     +    uploadpack.allowReachableSHA1InWant is set to true.
      
          Signed-off-by: Sun Chao <sunchao9@huawei.com>
      
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +~~~~~~~~~
      +
      +This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
     -+during the reference discovery phase, each reference and will be filtered
     ++during the reference discovery phase, each reference will be filtered
      +by this hook. The hook executes once with no arguments for each
      +'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
      +a version number and server process name ('uploadpack' or 'receive') will
      +send to it in pkt-line format, followed by a flush-pkt. The hook should
      +respond with its version number.
      +
     -+During reference discovery phase, each reference will be filtered by this
     -+hook. In the following example, the letter 'G' stands for 'git-receive-pack'
     -+or 'git-upload-pack' and the letter 'H' stands for this hook. The hook
     -+decides if the reference will be hidden or not, it sends result back in
     -+pkt-line format protocol, a response "hide" the references will hidden
     -+to the client.
     ++During the reference discovery phase, each reference will be filtered by
     ++this hook. In the following example, the letter 'G' stands for
     ++'git-receive-pack' or 'git-upload-pack', and the letter 'H' stands for
     ++this hook. The hook decides if the reference will be hidden or not, it
     ++sends the result back in pkt-line format protocol, and a response 'hide'
     ++means the references will be hidden to the client.
      +
      +	# Version negotiation
      +	G: PKT-LINE(version=1\0uploadpack)
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +	G: PKT-LINE(ref <refname>:<refnamefull>)
      +	G: flush-pkt
      +
     -+	# Receive result from the hook.
     ++	# Receive the result from the hook.
      +	# Case 1: this reference is hidden
      +	H: PKT-LINE(hide)
      +	H: flush-pkt
     @@ Documentation/githooks.txt: If this hook exits with a non-zero status, `git push
      +	# Case 2: this reference can be advertised
      +	H: flush-pkt
      +
     -+To enable the `hide-refs` hook, we should config hiderefs with `hook:`
     -+option, eg:
     ++To enable the `hide-refs` hook, we should config hiderefs with a `hook:`
     ++option, e.g. if we want to pass all the refs to the new hook except for
     ++the tags:
      +
     -+	git config --add transfer.hiderefs hook:refs/prefix1/
     -+	git config --add uploadpack.hiderefs hook:!refs/prefix2/
     ++	git config --add transfer.hiderefs hook:
     ++	git config --add transfer.hiderefs hook:!refs/tags/
      +
     -+the `hide-refs` will be called during reference discovery phase and
     ++the `hide-refs` will be called during the reference discovery phase and
      +check each matched reference, a 'hide' response means the reference will
     -+be hidden for its private data even if `allowTipSHA1InWant` and
     -+`allowReachableSHA1InWant` are set to true.
     ++be hidden for its private data even if `allowTipSHA1InWant` or
     ++`allowReachableSHA1InWant` is set to true.
      +
       [[pre-receive]]
       pre-receive

-- 
gitgitgadget

[PATCH v6 1/5] hiderefs: add hide-refs hook to hide refs dynamically

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Muti-branch workflows are used in some development scenarios, especially
for large teams, where different small teams are assigned to implement
different features on different branches or to develop secret features
on special branches.

If we can control the visible reference list based on developer
permissions, we can reduce the interference between reference lists of
different teams, and can achieve the protection of critical core code
(only certain team members can see it before making it public). This
kind of reference management makes sense, and on some platforms, sunch
as Gerrit implement it through server-side reference access control.

We can use '{transfer,uploadpack,receive}.hiderefs' config items to
control which references need to hide from clients, but the config items
are static and cannot satisfy the above requirements. We need the Git
server to hide references according to the user's permissions, we can
try to implement this mechanism by introducing a server-side hook
'hide-refs' to dynamically hide references during reference discovery
phase.

This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
during the reference discovery phase, each reference will be filtered
with this hook. The hook executes once with no arguments for each
'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
a version number and server process name ('uploadpack' or 'receive') will
send to it in pkt-line format, followed by a flush-pkt. The hook should
respond with its version number.

During reference discovery phase, each reference will be filtered by this
hook. In the following example, the letter 'G' stands for 'git-receive-pack'
or 'git-upload-pack', and the letter 'H' stands for this hook. The hook
decides if the reference will be hidden or not, and it sends the result
back in pkt-line format protocol, a response "hide" means the references
will be hidden to the client.

        # Version negotiation
        G: PKT-LINE(version=1\0uploadpack)
        G: flush-pkt
        H: PKT-LINE(version=1)
        H: flush-pkt

        # Send reference filter request to hook
        G: PKT-LINE(ref <refname>:<refname_full>)
        G: flush-pkt

        # Receive result from the hook.
        # Case 1: this reference is hidden
        H: PKT-LINE(hide)
        H: flush-pkt

        # Case 2: this reference can be advertised
        H: flush-pkt

To enable the `hide-refs` hook, we should config hiderefs with `hook:`
option, e.g. if we want to pass all the refs to the new hook except for
the tags:

        git config --add transfer.hiderefs hook:
        git config --add transfer.hiderefs hook:!refs/tags/

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 refs.c | 250 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 228 insertions(+), 22 deletions(-)

diff --git a/refs.c b/refs.c
index 92819732ab7..68368055946 100644
--- a/refs.c
+++ b/refs.c
@@ -8,6 +8,7 @@
 #include "lockfile.h"
 #include "iterator.h"
 #include "refs.h"
+#include "pkt-line.h"
 #include "refs/refs-internal.h"
 #include "run-command.h"
 #include "hook.h"
@@ -1384,39 +1385,221 @@ char *shorten_unambiguous_ref(const char *refname, int strict)
 }
 
 static struct string_list *hide_refs;
+static struct string_list *hook_hide_refs;
+static struct strbuf hide_refs_section = STRBUF_INIT;
 
 int parse_hide_refs_config(const char *var, const char *value, const char *section)
 {
+	struct string_list **refs_list= &hide_refs;
 	const char *key;
-	if (!strcmp("transfer.hiderefs", var) ||
-	    (!parse_config_key(var, section, NULL, NULL, &key) &&
-	     !strcmp(key, "hiderefs"))) {
-		char *ref;
-		int len;
-
-		if (!value)
-			return config_error_nonbool(var);
-		ref = xstrdup(value);
-		len = strlen(ref);
-		while (len && ref[len - 1] == '/')
-			ref[--len] = '\0';
-		if (!hide_refs) {
-			CALLOC_ARRAY(hide_refs, 1);
-			hide_refs->strdup_strings = 1;
+	char *ref;
+	int len;
+
+	if (strcmp("transfer.hiderefs", var) &&
+	    !(!parse_config_key(var, section, NULL, NULL, &key) &&
+	      !strcmp(key, "hiderefs")))
+		return 0;
+
+	if (!value)
+		return config_error_nonbool(var);
+
+	if (skip_prefix(value, "hook:", &value)) {
+		refs_list = &hook_hide_refs;
+
+		/*
+		 * Once the 'hide-refs' hook is invoked, Git needs to do
+		 * version negotiation with it, the version number and the
+		 * process name ('uploadpack' or 'receive') will send to
+		 * it in pkt-line format, and the process name is recorded
+		 * by hide_refs_section
+		 */
+		if (hide_refs_section.len == 0)
+			strbuf_addstr(&hide_refs_section, section);
+	}
+
+	ref = xstrdup(value);
+	len = strlen(ref);
+	while (len && ref[len - 1] == '/')
+		ref[--len] = '\0';
+
+	if (!*refs_list) {
+		CALLOC_ARRAY(*refs_list, 1);
+		(*refs_list)->strdup_strings = 1;
+	}
+	string_list_append(*refs_list, ref);
+
+	return 0;
+}
+
+static struct child_process *hide_refs_proc;
+static struct packet_reader *hide_refs_reader;
+
+/*
+ * Create the hide-refs hook child process and complete version negotiation,
+ * return non-zero upon success, otherwise 0
+ */
+static int create_hide_refs_process(void)
+{
+	struct child_process *proc;
+	struct packet_reader *reader;
+	const char *hook_path;
+	int version = 0;
+	int err;
+
+	hook_path = find_hook("hide-refs");
+	if (!hook_path)
+		return 0;
+
+	proc = (struct child_process *)xcalloc(1, sizeof (struct child_process));
+	reader = (struct packet_reader *)xcalloc(1, sizeof(struct packet_reader));
+
+	child_process_init(proc);
+	strvec_push(&proc->args, hook_path);
+	proc->in = -1;
+	proc->out = -1;
+	proc->trace2_hook_name = "hide-refs";
+	proc->err = 0;
+
+	err = start_command(proc);
+	if (err)
+		goto cleanup;
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+
+	/* Version negotiaton */
+	packet_reader_init(reader, proc->out, NULL, 0,
+			   PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+	err = packet_write_fmt_gently(proc->in, "version=1%c%s", '\0', hide_refs_section.buf);
+	if (!err)
+		err = packet_flush_gently(proc->in);
+
+	if (!err)
+		for (;;) {
+			enum packet_read_status status;
+
+			status = packet_reader_read(reader);
+			if (status != PACKET_READ_NORMAL) {
+				/* Check whether hide-refs exited abnormally */
+				if (status == PACKET_READ_EOF)
+					goto failure;
+				break;
+			}
+
+			if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+				version = atoi(reader->line + 8);
+			}
 		}
-		string_list_append(hide_refs, ref);
+
+	if (err)
+		goto failure;
+
+	switch (version) {
+	case 0:
+		/* fallthrough */
+	case 1:
+		break;
+	default:
+		trace_printf(_("hook hide-refs version '%d' is not supported"), version);
+		goto failure;
 	}
+
+	sigchain_pop(SIGPIPE);
+
+	hide_refs_proc = proc;
+	hide_refs_reader = reader;
+	return 1;
+
+failure:
+	close(proc->in);
+	close(proc->out);
+	kill(proc->pid, SIGTERM);
+	finish_command_in_signal(proc);
+
+cleanup:
+	free(proc);
+	free(reader);
+	sigchain_pop(SIGPIPE);
 	return 0;
 }
 
-int ref_is_hidden(const char *refname, const char *refname_full)
+/* If hide-refs child process start failed, set skip_hide_refs_proc to true */
+static int skip_hide_refs_proc;
+
+/*
+ * Return non-zero if hide-refs hook want to hide the ref and 0 otherwise,
+ * and return 0 if hide-refs child proccess start failed or exit abnormally
+ */
+static int ref_hidden_check_by_hook(const char *refname, const char *refname_full)
+{
+	struct strbuf buf = STRBUF_INIT;
+	int err;
+	int ret = 0;
+
+	if (skip_hide_refs_proc)
+		return 0;
+
+	if (!hide_refs_proc)
+		if (!create_hide_refs_process()) {
+			skip_hide_refs_proc = 1;
+			return 0;
+		}
+
+	sigchain_push(SIGPIPE, SIG_IGN);
+	err = packet_write_fmt_gently(hide_refs_proc->in, "ref %s:%s", refname, refname_full);
+	if (err)
+		goto cleanup;
+
+	err = packet_flush_gently(hide_refs_proc->in);
+	if (err)
+		goto cleanup;
+
+	for (;;) {
+		enum packet_read_status status;
+
+		status = packet_reader_read(hide_refs_reader);
+		if (status != PACKET_READ_NORMAL) {
+			/* Check whether hide-refs exited abnormally */
+			if (status == PACKET_READ_EOF)
+				goto cleanup;
+			break;
+		}
+
+		strbuf_addstr(&buf, hide_refs_reader->line);
+	}
+
+	if (!strncmp("hide", buf.buf, 4))
+		ret = 1;
+
+	sigchain_pop(SIGPIPE);
+	return ret;
+
+cleanup:
+	close(hide_refs_proc->in);
+	close(hide_refs_proc->out);
+	kill(hide_refs_proc->pid, SIGTERM);
+	finish_command_in_signal(hide_refs_proc);
+
+	free(hide_refs_proc);
+	free(hide_refs_reader);
+	sigchain_pop(SIGPIPE);
+
+	skip_hide_refs_proc = 1;
+	return 0;
+}
+
+static int ref_hidden_check(const char *refname, const char *refname_full, int hook)
 {
+	struct string_list *hide_refs_list = hide_refs;
 	int i;
+	int match_all = 0;
 
-	if (!hide_refs)
+	if (hook)
+		hide_refs_list = hook_hide_refs;
+	if (!hide_refs_list)
 		return 0;
-	for (i = hide_refs->nr - 1; i >= 0; i--) {
-		const char *match = hide_refs->items[i].string;
+
+	for (i = hide_refs_list->nr - 1; i >= 0; i--) {
+		const char *match = hide_refs_list->items[i].string;
 		const char *subject;
 		int neg = 0;
 		const char *p;
@@ -1426,6 +1609,12 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 			match++;
 		}
 
+		/* empty string with the 'hook:' option matches all the refs */
+		if (hook && !*match) {
+			match_all = !neg;
+			continue;
+		}
+
 		if (*match == '^') {
 			subject = refname_full;
 			match++;
@@ -1436,9 +1625,26 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 		/* refname can be NULL when namespaces are used. */
 		if (subject &&
 		    skip_prefix(subject, match, &p) &&
-		    (!*p || *p == '/'))
-			return !neg;
+		    (!*p || *p == '/')) {
+			if (neg)
+				return 0;
+			if (!hook)
+				return 1;
+			return ref_hidden_check_by_hook(refname, refname_full);
+		}
 	}
+
+	if (hook && match_all)
+		return ref_hidden_check_by_hook(refname, refname_full);
+
+	return 0;
+}
+
+int ref_is_hidden(const char *refname, const char *refname_full)
+{
+	if (ref_hidden_check(refname, refname_full, 0) ||
+	    ref_hidden_check(refname, refname_full, 1))
+		return 1;
 	return 0;
 }
 
-- 
gitgitgadget

[PATCH v6 2/5] hiderefs: use a new flag to mark force hidden refs

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

If uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
is set to true, the private commits of hiding refs can be fetched by
the client. The new hide-refs hook is used to hide our refs and we wish
to hide the private commits either.

A new flag `HIDDEN_REF_FORCE` is used to mark a ref if hide-refs hook
decides to hide it, and we make sure the wire protocol V1 will reject
to send the private commits of this kind of refs even if
uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
is set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 refs.c        | 39 +++++++++++++++++++++++++++++++++++++++
 refs.h        |  3 +++
 upload-pack.c | 14 --------------
 3 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/refs.c b/refs.c
index 68368055946..5a9079fd4c4 100644
--- a/refs.c
+++ b/refs.c
@@ -1648,6 +1648,45 @@ int ref_is_hidden(const char *refname, const char *refname_full)
 	return 0;
 }
 
+#define OUR_REF		(1u << 12)
+#define HIDDEN_REF	(1u << 19)
+/*
+ * Use this flag to mark a ref that is hidden by the hide-refs hook, its private
+ * commits (tip or non-tip commits, not reachable by the refs not hidden by the
+ * hide-refs hook) will be forced hidden to the client, which means a client can
+ * not fetch such kind of commits even uploadpack.allowTipSHA1InWant or
+ * uploadpack.allowReachableSHA1InWant are set to true
+ */
+#define HIDDEN_REF_FORCE	(1u << 20)
+
+static unsigned int ref_hidden_flag(const char *refname, const char *refname_full)
+{
+	if (ref_hidden_check(refname, refname_full, 1))
+		return HIDDEN_REF_FORCE;
+	else if (ref_hidden_check(refname, refname_full, 0))
+		return HIDDEN_REF;
+	return OUR_REF;
+}
+
+int mark_our_ref(const char *refname, const char *refname_full,
+		 const struct object_id *oid)
+{
+	struct object *o;
+	unsigned int flag;
+
+	if (!oid || is_null_oid(oid)) {
+		return 0;
+	}
+
+	o = lookup_unknown_object(the_repository, oid);
+	flag = ref_hidden_flag(refname, refname_full);
+	o->flags |= flag;
+
+	if (flag & OUR_REF)
+		return 0;
+	return 1;
+}
+
 const char *find_descendant_ref(const char *dirname,
 				const struct string_list *extras,
 				const struct string_list *skip)
diff --git a/refs.h b/refs.h
index d6575b8c2bd..2feabfe35c4 100644
--- a/refs.h
+++ b/refs.h
@@ -819,6 +819,9 @@ int parse_hide_refs_config(const char *var, const char *value, const char *);
  * parameter always points to the full ref name.
  */
 int ref_is_hidden(const char *, const char *);
+/* return non-zero if the ref is hidden, otherwise 0 */
+int mark_our_ref(const char *refname, const char *refname_full,
+		 const struct object_id *oid);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/upload-pack.c b/upload-pack.c
index b217a1f469e..a8ca5d1c26e 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1155,20 +1155,6 @@ static void receive_needs(struct upload_pack_data *data,
 		packet_flush(1);
 }
 
-/* return non-zero if the ref is hidden, otherwise 0 */
-static int mark_our_ref(const char *refname, const char *refname_full,
-			const struct object_id *oid)
-{
-	struct object *o = lookup_unknown_object(the_repository, oid);
-
-	if (ref_is_hidden(refname, refname_full)) {
-		o->flags |= HIDDEN_REF;
-		return 1;
-	}
-	o->flags |= OUR_REF;
-	return 0;
-}
-
 static int check_ref(const char *refname_full, const struct object_id *oid,
 		     int flag, void *cb_data)
 {
-- 
gitgitgadget

[PATCH v6 3/5] hiderefs: hornor hide flags in wire protocol V2

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Previously hiderefs configurations can not protect the private data
of hiding refs in wire protocol V2, for example, an `ALL_FLAGS` flag
will be used to clear all the objects before handling the fetch requests.

Hornor the hide flags by removing the `HIDDEN_REFS` flag from the
`ALL_FLAGS` and make sure all the refs will check its hidden flags
before sending the pack to client, especially during stateless RPC.
And if there are refs with `HIDDEN_REF_FORCE` flag, use `check_non_tip`
to protect the private data of force-hidden refs.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 ls-refs.c     |  2 +-
 refs.c        | 20 ++++++++++++++++++++
 refs.h        |  4 ++++
 upload-pack.c | 11 +++++++++--
 4 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/ls-refs.c b/ls-refs.c
index 98e69373c84..b5cb1316d38 100644
--- a/ls-refs.c
+++ b/ls-refs.c
@@ -84,7 +84,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
 
 	strbuf_reset(&data->buf);
 
-	if (ref_is_hidden(refname_nons, refname))
+	if (mark_our_ref(refname_nons, refname, oid))
 		return 0;
 
 	if (!ref_match(&data->prefixes, refname_nons))
diff --git a/refs.c b/refs.c
index 5a9079fd4c4..847f7c003e6 100644
--- a/refs.c
+++ b/refs.c
@@ -1659,6 +1659,25 @@ int ref_is_hidden(const char *refname, const char *refname_full)
  */
 #define HIDDEN_REF_FORCE	(1u << 20)
 
+/* Use this variable to record existing object hidden flags */
+static unsigned int objects_hidden_flags;
+
+/* Return non-zero if need to batch check hidden refs, otherwise 0 */
+int need_check_hidden_refs(void)
+{
+	if (!objects_hidden_flags)
+		return 1;
+	return 0;
+}
+
+/* Return non-zero if some ref is force hidden, otherwise 0 */
+int has_force_hidden_refs(void)
+{
+	if (objects_hidden_flags & HIDDEN_REF_FORCE)
+		return 1;
+	return 0;
+}
+
 static unsigned int ref_hidden_flag(const char *refname, const char *refname_full)
 {
 	if (ref_hidden_check(refname, refname_full, 1))
@@ -1681,6 +1700,7 @@ int mark_our_ref(const char *refname, const char *refname_full,
 	o = lookup_unknown_object(the_repository, oid);
 	flag = ref_hidden_flag(refname, refname_full);
 	o->flags |= flag;
+	objects_hidden_flags |= flag;
 
 	if (flag & OUR_REF)
 		return 0;
diff --git a/refs.h b/refs.h
index 2feabfe35c4..8deb36a95cc 100644
--- a/refs.h
+++ b/refs.h
@@ -822,6 +822,10 @@ int ref_is_hidden(const char *, const char *);
 /* return non-zero if the ref is hidden, otherwise 0 */
 int mark_our_ref(const char *refname, const char *refname_full,
 		 const struct object_id *oid);
+/* return non-zero if need to batch check hidden refs, otherwise 0 */
+int need_check_hidden_refs(void);
+/* return non-zero if some ref is force hidden, otherwise 0 */
+int has_force_hidden_refs(void);
 
 enum ref_type {
 	REF_TYPE_PER_WORKTREE,	  /* refs inside refs/ but not shared       */
diff --git a/upload-pack.c b/upload-pack.c
index a8ca5d1c26e..a9a24399d8e 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -39,8 +39,8 @@
 #define CLIENT_SHALLOW	(1u << 18)
 #define HIDDEN_REF	(1u << 19)
 
-#define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \
-		NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF)
+#define ALL_FLAGS (THEY_HAVE |WANTED | COMMON_KNOWN | SHALLOW | \
+		NOT_SHALLOW | CLIENT_SHALLOW)
 
 /* Enum for allowed unadvertised object request (UOR) */
 enum allow_uor {
@@ -1726,6 +1726,13 @@ int upload_pack_v2(struct repository *r, struct packet_reader *request)
 				state = FETCH_DONE;
 			break;
 		case FETCH_SEND_PACK:
+			if (need_check_hidden_refs()) {
+				head_ref_namespaced(check_ref, NULL);
+				for_each_namespaced_ref(check_ref, NULL);
+			}
+			if (has_force_hidden_refs())
+				check_non_tip(&data);
+
 			send_wanted_ref_info(&data);
 			send_shallow_info(&data);
 
-- 
gitgitgadget

[PATCH v6 4/5] test: add test cases for hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

Add test cases for the new 'hide-refs' hook which is used to
filter the references during the reference discovery phase.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Makefile                                      |   1 +
 t/helper/test-hide-refs.c                     | 107 +++++++++++++++++
 t/helper/test-tool.c                          |   1 +
 t/helper/test-tool.h                          |   1 +
 t/t1419-hide-refs-hook.sh                     | 111 ++++++++++++++++++
 t/t1419/abnormal-hide-refs-hook.sh            |  80 +++++++++++++
 t/t1419/common-functions.sh                   |  74 ++++++++++++
 ...test-0001-ls-remote-with-hide-refs-hook.sh |  43 +++++++
 ...st-0002-upload-pack-with-hide-refs-hook.sh |  45 +++++++
 ...t-0003-receive-pack-with-hide-refs-hook.sh |  40 +++++++
 10 files changed, 503 insertions(+)
 create mode 100644 t/helper/test-hide-refs.c
 create mode 100755 t/t1419-hide-refs-hook.sh
 create mode 100644 t/t1419/abnormal-hide-refs-hook.sh
 create mode 100644 t/t1419/common-functions.sh
 create mode 100644 t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
 create mode 100644 t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh

diff --git a/Makefile b/Makefile
index 924b864ae83..c6793681b41 100644
--- a/Makefile
+++ b/Makefile
@@ -794,6 +794,7 @@ TEST_BUILTINS_OBJS += test-wildmatch.o
 TEST_BUILTINS_OBJS += test-windows-named-pipe.o
 TEST_BUILTINS_OBJS += test-write-cache.o
 TEST_BUILTINS_OBJS += test-xml-encode.o
+TEST_BUILTINS_OBJS += test-hide-refs.o
 
 # Do not add more tests here unless they have extra dependencies. Add
 # them in TEST_BUILTINS_OBJS above.
diff --git a/t/helper/test-hide-refs.c b/t/helper/test-hide-refs.c
new file mode 100644
index 00000000000..751fc6213f3
--- /dev/null
+++ b/t/helper/test-hide-refs.c
@@ -0,0 +1,107 @@
+#include "cache.h"
+#include "hash.h"
+#include "config.h"
+#include "connect.h"
+#include "parse-options.h"
+#include "pkt-line.h"
+#include "sigchain.h"
+#include "test-tool.h"
+
+static const char *hide_refs_usage[] = {
+	"test-tool hide-refs [<options>...]",
+	NULL
+};
+
+static int die_before_read_ref;
+static int die_after_proc_ref;
+static int version = 1;
+static int hash_size = GIT_SHA1_HEXSZ;
+static struct string_list hidelist = STRING_LIST_INIT_NODUP;
+
+static void hide_refs_verison(struct packet_reader *reader) {
+	int server_version = 0;
+
+	for (;;) {
+		if (packet_reader_read(reader) != PACKET_READ_NORMAL)
+			break;
+
+		if (reader->pktlen > 8 && starts_with(reader->line, "version=")) {
+			server_version = atoi(reader->line+8);
+			if (server_version != 1)
+				die("bad protocol version: %d", server_version);
+		}
+	}
+
+	packet_write_fmt(1, "version=%d\n", version);
+	packet_flush(1);
+}
+
+static void hide_refs_proc(struct packet_reader *reader)
+{
+	const char *p;
+	struct strbuf buf = STRBUF_INIT;
+	enum packet_read_status status;
+
+	if (die_before_read_ref)
+		die("die with the --die-before-read-ref option");
+
+	for (;;) {
+		status = packet_reader_read(reader);
+		if (status == PACKET_READ_EOF)
+			exit(0);
+
+		if (status != PACKET_READ_NORMAL)
+			break;
+
+		p = reader->line;
+		strbuf_reset(&buf);
+		strbuf_addstr(&buf, reader->line);
+	}
+
+	p = strchr(buf.buf, ':');
+	if (unsorted_string_list_has_string(&hidelist, p + 1)) {
+		packet_write_fmt(1, "hide");
+	}
+
+	if (die_after_proc_ref)
+		die("die with the --die-after-proc-refs option");
+
+	packet_flush(1);
+}
+
+int cmd__hide_refs(int argc, const char **argv) {
+	int nongit_ok = 0;
+	struct packet_reader reader;
+	const char *value = NULL;
+	struct option options[] = {
+		OPT_BOOL(0, "die-before-read-ref", &die_before_read_ref,
+			 "die when reading first reference"),
+		OPT_BOOL(0, "die-after-proc-refs", &die_after_proc_ref,
+			 "die after proc ref"),
+		OPT_STRING_LIST('H', "hide", &hidelist, "refs-to-force-hidden",
+				"refs that will be force hidden"),
+		OPT_INTEGER('V', "version", &version,
+			    "use this protocol version number"),
+		OPT_END()
+	};
+
+	setup_git_directory_gently(&nongit_ok);
+
+	argc = parse_options(argc, argv, "test-tools", options, hide_refs_usage, 0);
+	if (argc > 0)
+		usage_msg_opt("Too many arguments.", hide_refs_usage, options);
+
+	packet_reader_init(&reader, 0, NULL, 0, PACKET_READ_CHOMP_NEWLINE | PACKET_READ_GENTLE_ON_EOF);
+
+	if (!git_config_get_value("extensions.objectformat", &value)) {
+		if (!strcmp(value, "sha256"))
+			hash_size = GIT_SHA256_HEXSZ;
+	}
+
+	hide_refs_verison(&reader);
+	for (;;) {
+		hide_refs_proc(&reader);
+	}
+
+	return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index 80055886798..c5bd7f1f806 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -72,6 +72,7 @@ static struct test_cmd cmds[] = {
 	{ "regex", cmd__regex },
 	{ "repository", cmd__repository },
 	{ "revision-walking", cmd__revision_walking },
+	{ "hide-refs", cmd__hide_refs },
 	{ "run-command", cmd__run_command },
 	{ "scrap-cache-tree", cmd__scrap_cache_tree },
 	{ "serve-v2", cmd__serve_v2 },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index a432cc77d92..cba8d3b093d 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -61,6 +61,7 @@ int cmd__reftable(int argc, const char **argv);
 int cmd__regex(int argc, const char **argv);
 int cmd__repository(int argc, const char **argv);
 int cmd__revision_walking(int argc, const char **argv);
+int cmd__hide_refs(int argc, const char **argv);
 int cmd__run_command(int argc, const char **argv);
 int cmd__scrap_cache_tree(int argc, const char **argv);
 int cmd__serve_v2(int argc, const char **argv);
diff --git a/t/t1419-hide-refs-hook.sh b/t/t1419-hide-refs-hook.sh
new file mode 100755
index 00000000000..81a8c36190b
--- /dev/null
+++ b/t/t1419-hide-refs-hook.sh
@@ -0,0 +1,111 @@
+#!/bin/sh
+#
+# Copyright (c) 2022 Sun Chao
+#
+
+test_description='Test hide-refs hook'
+
+. ./test-lib.sh
+. "$TEST_DIRECTORY"/t1419/common-functions.sh
+
+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+
+setup_test_repos () {
+	test_expect_success "setup bare_repo and work_repo" '
+		rm -rf bare_repo.git &&
+		rm -rf work_repo &&
+		git init --bare bare_repo.git &&
+		git init work_repo &&
+
+		# create new commits and references
+		create_commits_in work_repo A B C D &&
+		(
+			cd work_repo &&
+			git config --local core.abbrev 7 &&
+			git update-ref refs/heads/main $A &&
+			git update-ref refs/heads/dev $B &&
+			git update-ref refs/pull-requests/1/head $C &&
+			git tag -m "v123" v123 $D &&
+			git push ../bare_repo.git +refs/heads/*:refs/heads/* &&
+			git push ../bare_repo.git +refs/tags/*:refs/tags/* &&
+			git push ../bare_repo.git +refs/pull-requests/*:refs/pull-requests/*
+		) &&
+		TAG=$(git -C work_repo rev-parse v123) &&
+
+		# config transfer.hiderefs values with "hook:" prefix
+		(
+			git -C bare_repo.git config --local http.receivepack true &&
+			git -C bare_repo.git config --add transfer.hiderefs hook:
+		)
+	'
+}
+
+setup_httpd() {
+	ROOT_PATH="$PWD"
+	. "$TEST_DIRECTORY"/lib-gpg.sh
+	. "$TEST_DIRECTORY"/lib-httpd.sh
+	. "$TEST_DIRECTORY"/lib-terminal.sh
+
+	start_httpd
+	set_askpass user@host pass@host
+	setup_askpass_helper
+}
+
+# Run test cases when hide-refs hook exit abnormally
+run_tests_for_abnormal_hook() {
+	GIT_TEST_PROTOCOL_VERSION=$1
+	BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+
+	for t in  "$TEST_DIRECTORY"/t1419/abnormal-*.sh
+	do
+		setup_test_repos
+
+		. "$t"
+	done
+}
+
+# Run test cases under local/HTTP protocol
+run_tests_for_normal_hook() {
+	for t in  "$TEST_DIRECTORY"/t1419/test-*.sh
+	do
+		setup_test_repos
+		case $1 in
+			http)
+				PROTOCOL="HTTP protocol"
+
+				# bare_repo.git need move to httpd sever root path
+				BAREREPO_GIT_DIR="$HTTPD_DOCUMENT_ROOT_PATH/bare_repo.git"
+				rm -rf "$BAREREPO_GIT_DIR"
+				mv bare_repo.git "$BAREREPO_GIT_DIR"
+
+				# setup the repository service URL address of http protocol
+				BAREREPO_PREFIX="$HTTPD_URL"/smart
+				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+				;;
+			local)
+				PROTOCOL="builtin protocol"
+				BAREREPO_GIT_DIR="$(pwd)/bare_repo.git"
+
+				# setup the repository service address of builtin protocol
+				BAREREPO_PREFIX="$(pwd)"
+				BAREREPO_URL="$BAREREPO_PREFIX/bare_repo.git"
+				;;
+		esac
+
+		GIT_TEST_PROTOCOL_VERSION=$2
+		git -C work_repo remote add origin "$BAREREPO_URL"
+
+		. "$t"
+	done
+}
+
+setup_httpd
+for protocol in 1 2
+do
+	run_tests_for_abnormal_hook $protocol
+	run_tests_for_normal_hook local $protocol
+	run_tests_for_normal_hook http $protocol
+done
+
+test_done
diff --git a/t/t1419/abnormal-hide-refs-hook.sh b/t/t1419/abnormal-hide-refs-hook.sh
new file mode 100644
index 00000000000..1ce768191c8
--- /dev/null
+++ b/t/t1419/abnormal-hide-refs-hook.sh
@@ -0,0 +1,80 @@
+#!/bin/sh
+
+# The upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer] hiderefs = hook:
+#
+# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it,
+# we should make sure Git works correctly in some special cases
+
+# If the hide-refs does not exist, Git should not invoke it and continue to advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook not exists" '
+	rm -f "$BAREREPO_GIT_DIR/hooks/hide-refs" &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If the hide-refs hook run with incompatible version, Git should not invoke it and continue to advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook run with incompatible version" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --version=2
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If the hide-refs hook exit before processing any refs, Git should not die and continue to advertise all the refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die before read ref" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --die-before-read-ref
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-before-read-ref option
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If the hide-refs hook exit abnormally, Git should not die and continue to advertise left refs
+test_expect_success "protocol $GIT_TEST_PROTOCOL_VERSION: advertise-refs while hide-refs hook die after proc ref" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs --die-after-proc-refs
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION upload-pack --advertise-refs "$BAREREPO_GIT_DIR" >out 2>&1 &&
+	cat out | make_user_friendly_and_stable_output | grep -v "^error:" >actual &&
+	format_and_save_expect <<-EOF &&
+		fatal: die with the --die-after-proc-refs option
+		<COMMIT-A> HEAD
+		<COMMIT-B> refs/heads/dev
+		<COMMIT-A> refs/heads/main
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/common-functions.sh b/t/t1419/common-functions.sh
new file mode 100644
index 00000000000..7841241b038
--- /dev/null
+++ b/t/t1419/common-functions.sh
@@ -0,0 +1,74 @@
+# Create commits in <repo> and assign each commit's oid to shell variables
+# given in the arguments (A, B, and C). E.g.:
+#
+#     create_commits_in <repo> A B C
+#
+# NOTE: Never calling this function from a subshell since variable
+# assignments will disappear when subshell exits.
+create_commits_in () {
+	repo="$1" && test -d "$repo" ||
+	error "Repository $repo does not exist."
+	shift &&
+	while test $# -gt 0
+	do
+		name=$1 &&
+		shift &&
+		test_commit -C "$repo" --no-tag "$name" &&
+		eval $name=$(git -C "$repo" rev-parse HEAD)
+	done
+}
+
+get_abbrev_oid () {
+	oid=$1 &&
+	suffix=${oid#???????} &&
+	oid=${oid%$suffix} &&
+	if test -n "$oid"
+	then
+		echo "$oid"
+	else
+		echo "undefined-oid"
+	fi
+}
+
+# Format the output of git-fetch, git-ls-remote and other commands to make a
+# user-friendly and stable text.  We can easily prepare the expect text
+# without having to worry about changes of the commit ID (full or abbrev.)
+# of the output.  Single quotes are replaced with double quotes, because
+# it is boring to prepare unquoted single quotes in expect text.
+make_user_friendly_and_stable_output () {
+	tr '\0' '@' | sed \
+		-e "s/'/\"/g" \
+		-e "s/@.*//g" \
+		-e "s/$(get_abbrev_oid $A)[0-9a-f]*/<COMMIT-A>/g" \
+		-e "s/$(get_abbrev_oid $B)[0-9a-f]*/<COMMIT-B>/g" \
+		-e "s/$(get_abbrev_oid $C)[0-9a-f]*/<COMMIT-C>/g" \
+		-e "s/$(get_abbrev_oid $D)[0-9a-f]*/<COMMIT-D>/g" \
+		-e "s/$(get_abbrev_oid $TAG)[0-9a-f]*/<COMMIT-TAG-v123>/g" \
+		-e "s/$ZERO_OID/<ZERO-OID>/g" \
+		-e "s#$BAREREPO_PREFIX/bare_repo.git#<URL/of/bare_repo.git>#" \
+		-e 's/^[0-9a-f]\{4\}//g'
+
+}
+
+filter_out_hide_refs_output() {
+	make_user_friendly_and_stable_output | sed 's/^[0-9a-f]\{4\}//g'
+}
+
+format_and_save_expect () {
+	sed -e 's/^> //' -e 's/Z$//' >expect
+}
+
+test_cmp_refs () {
+	indir=
+	if test "$1" = "-C"
+	then
+		shift
+		indir="$1"
+		shift
+	fi
+	indir=${indir:+"$indir"/}
+	cat >show-ref.expect &&
+	git ${indir:+ -C "$indir"} show-ref >show-ref.pristine &&
+	make_user_friendly_and_stable_output <show-ref.pristine >show-ref.filtered &&
+	test_cmp show-ref.expect show-ref.filtered
+}
diff --git a/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..3831521d345
--- /dev/null
+++ b/t/t1419/test-0001-ls-remote-with-hide-refs-hook.sh
@@ -0,0 +1,43 @@
+#!/bin/sh
+
+# The upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer] hiderefs = hook:
+#
+# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
+
+# Git will not advertise the refs that are hidden by the hide-refs hook
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide part of refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/pull-requests/1/head" \
+		-H "refs/tags/v123"
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A>	HEAD
+		<COMMIT-B>	refs/heads/dev
+		<COMMIT-A>	refs/heads/main
+	EOF
+	test_cmp expect actual
+'
+
+# The hide-ref hook should not change the default effects of '{transfer,uploadpack,receive}.hiderefs'
+# configurations, if it hides no refs, the original hiderefs configurations should work
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): ls-remote while hide-refs hook hide no refs" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs
+	EOF
+	git -C "$BAREREPO_GIT_DIR" config --add transfer.hiderefs refs/heads/dev &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION ls-remote "$BAREREPO_URL" >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-A>	HEAD
+		<COMMIT-A>	refs/heads/main
+		<COMMIT-C>	refs/pull-requests/1/head
+		<COMMIT-TAG-v123>	refs/tags/v123
+		<COMMIT-D>	refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
diff --git a/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..618a54e16a1
--- /dev/null
+++ b/t/t1419/test-0002-upload-pack-with-hide-refs-hook.sh
@@ -0,0 +1,45 @@
+#!/bin/sh
+
+# The upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer] hiderefs = hook:
+#
+# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
+
+# Git client can not fetch the refs that are hidden by the hide-refs hook
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): mirror clone while hide-refs hide part of refs" '
+	rm -rf local.git &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "HEAD" \
+		-H "refs/heads/dev" \
+		-H "refs/heads/main"
+	EOF
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION clone --mirror "$BAREREPO_URL" local.git &&
+	git -C local.git show-ref -d >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >actual &&
+	format_and_save_expect <<-EOF &&
+		<COMMIT-C> refs/pull-requests/1/head
+		<COMMIT-TAG-v123> refs/tags/v123
+		<COMMIT-D> refs/tags/v123^{}
+	EOF
+	test_cmp expect actual
+'
+
+# If a ref is hidden by the hide-refs hook, its private commits (tip or non-tip) will be forced hidden
+# to the client, and the client can not fetch such kind of commits even if the server set allowTipSHA1InWant
+# or allowReachableSHA1InWant to true
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): fetch a commit which is hided by hide-refs hook" '
+	rm -rf local.git &&
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/heads/dev" \
+		-H "refs/pull-requests/1/head" \
+		-H "refs/tags/v123"
+	EOF
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowTipSHA1InWant true &&
+	git -C "$BAREREPO_GIT_DIR" config uploadpack.allowReachableSHA1InWant true &&
+	git init local.git &&
+	git -C local.git remote add origin "$BAREREPO_URL" &&
+	test_must_fail git -C local.git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION fetch "$BAREREPO_URL" $B
+'
diff --git a/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
new file mode 100644
index 00000000000..11f0f255b31
--- /dev/null
+++ b/t/t1419/test-0003-receive-pack-with-hide-refs-hook.sh
@@ -0,0 +1,40 @@
+#!/bin/sh
+
+# The upstream repository (bare_repo.git) contains the configurations:
+#
+#	[transfer] hiderefs = hook:
+#
+# During the reference advertise phase the hide-refs hook will be invoked and all the refs will be checked by it
+
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook does not hide it" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+		test-tool hide-refs
+	EOF
+	create_commits_in work_repo E &&
+	git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		To <URL/of/bare_repo.git>
+		   <COMMIT-A>..<COMMIT-E>  HEAD -> main
+	EOF
+	test_cmp expect actual
+'
+
+# If hide-refs hook hide some ref, git push will be rejected
+test_expect_success "$PROTOCOL (protocol: $GIT_TEST_PROTOCOL_VERSION): push to main while hide-refs hook hide it" '
+	write_script "$BAREREPO_GIT_DIR/hooks/hide-refs" <<-EOF &&
+	test-tool hide-refs \
+		-H "refs/heads/main"
+	EOF
+	create_commits_in work_repo F &&
+	test_must_fail git -c protocol.version=$GIT_TEST_PROTOCOL_VERSION -C work_repo push origin HEAD:main >out 2>&1 &&
+	make_user_friendly_and_stable_output <out >out.tmp &&
+	sed "s/$(get_abbrev_oid $E)[0-9a-f]*/<COMMIT-E>/g" <out.tmp >actual &&
+	format_and_save_expect <<-EOF &&
+		To <URL/of/bare_repo.git>
+		 ! [remote rejected] HEAD -> main (deny updating a hidden ref)
+		error: failed to push some refs to "<URL/of/bare_repo.git>"
+	EOF
+	test_cmp expect actual
+'
-- 
gitgitgadget

[PATCH v6 5/5] doc: add documentation for the hide-refs hook

[Sun Chao via GitGitGadget]

From: Sun Chao <sunchao9@huawei.com>

If uploadpack.allowTipSHA1InWant or uploadpack.allowReachableSHA1InWant
is set to true, the private commits of hiding refs can be fetched by
the client. The new "hide-refs" hook is used to hide our refs and we
wish to hide the private commits either.

"git-upload-pack" or "git-receive-pack" can use "hide-refs" hook to
filter the references during the reference discovery phase. If a ref
is hidden by the "hide-refs" hook, its private data cannot be fetched
by the client even if uploadpack.allowTipSHA1InWant or
uploadpack.allowReachableSHA1InWant is set to true.

Signed-off-by: Sun Chao <sunchao9@huawei.com>
---
 Documentation/githooks.txt | 49 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index a16e62bc8c8..b26e50a4ea7 100644
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -249,6 +249,55 @@ If this hook exits with a non-zero status, `git push` will abort without
 pushing anything.  Information about why the push is rejected may be sent
 to the user by writing to standard error.
 
+[[hide-refs]]
+hide-refs
+~~~~~~~~~
+
+This hook would be invoked by 'git-receive-pack' and 'git-upload-pack'
+during the reference discovery phase, each reference will be filtered
+by this hook. The hook executes once with no arguments for each
+'git-upload-pack' and 'git-receive-pack' process. Once the hook is invoked,
+a version number and server process name ('uploadpack' or 'receive') will
+send to it in pkt-line format, followed by a flush-pkt. The hook should
+respond with its version number.
+
+During the reference discovery phase, each reference will be filtered by
+this hook. In the following example, the letter 'G' stands for
+'git-receive-pack' or 'git-upload-pack', and the letter 'H' stands for
+this hook. The hook decides if the reference will be hidden or not, it
+sends the result back in pkt-line format protocol, and a response 'hide'
+means the references will be hidden to the client.
+
+	# Version negotiation
+	G: PKT-LINE(version=1\0uploadpack)
+	G: flush-pkt
+	H: PKT-LINE(version=1)
+	H: flush-pkt
+
+	# Send reference filter request to hook
+	G: PKT-LINE(ref <refname>:<refnamefull>)
+	G: flush-pkt
+
+	# Receive the result from the hook.
+	# Case 1: this reference is hidden
+	H: PKT-LINE(hide)
+	H: flush-pkt
+
+	# Case 2: this reference can be advertised
+	H: flush-pkt
+
+To enable the `hide-refs` hook, we should config hiderefs with a `hook:`
+option, e.g. if we want to pass all the refs to the new hook except for
+the tags:
+
+	git config --add transfer.hiderefs hook:
+	git config --add transfer.hiderefs hook:!refs/tags/
+
+the `hide-refs` will be called during the reference discovery phase and
+check each matched reference, a 'hide' response means the reference will
+be hidden for its private data even if `allowTipSHA1InWant` or
+`allowReachableSHA1InWant` is set to true.
+
 [[pre-receive]]
 pre-receive
 ~~~~~~~~~~~
-- 
gitgitgadget