git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Teng Long <dyroneteng@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, avarab@gmail.com, jonathantanmy@google.com,
	bagasdotme@gmail.com, adlternative@gmail.com, stolee@gmail.com,
	Teng Long <dyroneteng@gmail.com>
Subject: [PATCH v6 04/12] Introduce `uploadpack.excludeobject` configuration
Date: Tue, 19 Oct 2021 19:38:28 +0800	[thread overview]
Message-ID: <b1d779b26a2f5d646b09131645b503d2e82a29ec.1634634814.git.tenglong@alibaba-inc.com> (raw)
In-Reply-To: <cover.1634634814.git.tenglong@alibaba-inc.com>

From: Teng Long <dyroneteng@gmail.com>

Background:

The `uploadpack.blobpackfileuri` is made for the "packfile-uri" feature,
the feature is similar to a CDN cache, it supports the client to download
 pack file by a URI directly, without or reducing the server load when make
packing and transporting.

Reasons:

First reason, `uploadpack.blobpackfileuri` supports to exclude a single
object, but actually the object type can be not only a blob, so the name
is inaccurate currently.

Secondly, the name of the old configuration is not abstract enough, this
make the furthur extension difficult. If do not change its name, to let
different object types use different configuration names, the
configuration items will be bloated and difficult to maintain, so the
new configuration is more abstract in name.

Configuation format diff:

	old: uploadpack.blobPackfileUri=<object-hash> <pack-hash> <uri>
	new: uploadpack.excludeobject=<object-hash> <level> <pack-hash> <uri>

The new configuration `uploadpack.excludeobject` not only supports to
exclude a single object itself, but also to exclude related objects with
it at once, the scope of exclusion is determined by the object type and
the specified `<level>` value in the entry.

-Compatibility:

Although a new configuration has been introduced, the old one is
available to use and compatible with the new configuration.

Signed-off-by: Teng Long <dyroneteng@gmail.com>
---
 builtin/pack-objects.c | 33 ++++++++++++++++++++++++++++++++-
 upload-pack.c          |  7 +++++++
 2 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index a41a0a3ea7..75461483c0 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -3061,6 +3061,36 @@ static int git_pack_config(const char *k, const char *v, void *cb)
 		ex->uri = xstrdup(pack_end + 1);
 		oidmap_put(&configured_exclusions, ex);
 	}
+	if (!strcmp(k, "uploadpack.excludeobject")) {
+		struct configured_exclusion *ex = xmalloc(sizeof(*ex));
+		const char *oid_end, *pack_end, *type_end;
+		struct object_id pack_hash;
+		char type[2];
+		int level;
+
+		if (parse_oid_hex(v, &ex->e.oid, &oid_end) ||
+		    *oid_end != ' ' ||
+		    !strlcpy(type, oid_end + 1, sizeof(type)) ||
+		    parse_oid_hex(oid_end + 3, &pack_hash, &pack_end) ||
+		    *pack_end != ' ')
+			die(_("value of uploadpack.excludeobject must be "
+			      "of the form '<object-hash> <level> <pack-hash> <uri>' (got '%s')"), v);
+		if (oidmap_get(&configured_exclusions, &ex->e.oid))
+			die(_("object already configured by an earlier "
+			      "uploadpack.excludeobject (got '%s')"), v);
+
+		level = atoi(type);
+		if (level < ET_SELF || level > ET_REACHABLE) {
+			die(_("value of <level> must be 0 or 1 or 2 (got '%s')"), v);
+		}
+		ex->level = level;
+		type_end = oid_end + 2;
+		ex->pack_hash_hex = xcalloc(1, pack_end - type_end);
+		memcpy(ex->pack_hash_hex, type_end + 1, pack_end - type_end - 1);
+		ex->uri = xstrdup(pack_end + 1);
+		oidmap_put(&configured_exclusions, ex);
+	}
+
 	return git_default_config(k, v, cb);
 }
 
@@ -3887,7 +3917,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 			 N_("respect islands during delta compression")),
 		OPT_STRING_LIST(0, "uri-protocol", &uri_protocols,
 				N_("protocol"),
-				N_("exclude any configured uploadpack.blobpackfileuri with this protocol")),
+				N_("exclude any configured uploadpack.excludeobject or "
+					    "uploadpack.blobpackfileuri with this protocol")),
 		OPT_END(),
 	};
 
diff --git a/upload-pack.c b/upload-pack.c
index 5c1cd19612..d26fb351a3 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1751,6 +1751,13 @@ int upload_pack_advertise(struct repository *r,
 			strbuf_addstr(value, " packfile-uris");
 			free(str);
 		}
+
+		if (!repo_config_get_string(the_repository,
+					    "uploadpack.excludeobject",
+					    &str) && str) {
+			strbuf_addstr(value, " packfile-uris");
+			free(str);
+		}
 	}
 
 	return 1;
-- 
2.31.1.453.g945ddc3a74.dirty


  parent reply	other threads:[~2021-10-19 11:39 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-07  2:11 [PATCH] Packfile-uris support excluding commit objects Teng Long
2021-05-10 11:14 ` Ævar Arnfjörð Bjarmason
2021-05-18  8:49 ` [PATCH v2 0/3] packfile-uris: commit objects exclusion Teng Long
2021-05-18  8:49   ` [PATCH v2 1/3] packfile-uris: support for excluding commit object Teng Long
2021-05-19  4:28     ` Junio C Hamano
2021-05-20  4:46     ` Junio C Hamano
2021-05-18  8:49   ` [PATCH v2 2/3] packfile-uris.txt: " Teng Long
2021-05-18  8:49   ` [PATCH v2 3/3] t5702: excluding commits with packfile-uris Teng Long
2021-07-26  9:46   ` [PATCH v3 0/3] packfile-uris: commit objects exclusio Teng Long
2021-07-26  9:46     ` [PATCH v3 1/3] packfile-uris: support for excluding commit objects Teng Long
2021-07-26 18:15       ` Junio C Hamano
2021-07-26 19:45         ` Felipe Contreras
2021-08-11  1:44         ` Teng Long
2021-07-26  9:46     ` [PATCH v3 2/3] t5702: " Teng Long
2021-07-26 15:03       ` Ævar Arnfjörð Bjarmason
2021-08-11  1:46         ` [PATCH v3 1/3] packfile-uris: " Teng Long
2021-07-26  9:46     ` [PATCH v3 3/3] packfile-uri.txt: " Teng Long
2021-07-26 20:52       ` Junio C Hamano
2021-08-11  1:47         ` Teng Long
2021-07-26 12:34     ` [PATCH v3 0/3] packfile-uris: commit objects exclusio Ævar Arnfjörð Bjarmason
2021-08-11  1:48       ` Teng Long
2021-08-11  7:45     ` [PATCH v4 0/7] packfile-uris: commits and trees exclusion Teng Long
2021-08-11  7:45       ` [PATCH v4 1/7] pack-objects.c: introduce new method `match_packfile_uri_exclusions` Teng Long
2021-08-11  7:45       ` [PATCH v4 2/7] Add new parameter "carry_data" for "show_object" function Teng Long
2021-08-11  7:45       ` [PATCH v4 3/7] packfile-uri: support for excluding commit objects Teng Long
2021-08-11  7:45       ` [PATCH v4 4/7] packfile-uri: support for excluding tree objects Teng Long
2021-08-11  7:45       ` [PATCH v4 5/7] packfile-uri.txt: support for excluding commits and trees Teng Long
2021-08-11  9:59         ` Bagas Sanjaya
2021-08-11  7:45       ` [PATCH v4 6/7] t5702: replace with "test_when_finished" for cleanup Teng Long
2021-08-11  7:45       ` [PATCH v4 7/7] t5702: support for excluding commit objects Teng Long
2021-08-25  2:21       ` [PATCH v5 00/14] packfile-uris: commits, trees and tags exclusion Teng Long
2021-08-25  2:21         ` [PATCH v5 01/14] pack-objects.c: introduce new method `match_packfile_uri_exclusions` Teng Long
2021-08-25  2:21         ` [PATCH v5 02/14] Add new parameter "carry_data" for "show_object" function Teng Long
2021-08-26 20:45           ` Junio C Hamano
2021-09-02 11:08             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 03/14] packfile-uri: support for excluding commit objects Teng Long
2021-08-25 23:49           ` Ævar Arnfjörð Bjarmason
2021-09-02 12:26             ` Teng Long
2021-08-26 20:56           ` Junio C Hamano
2021-09-02 12:51             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 04/14] packfile-uri: support for excluding tree objects Teng Long
2021-08-25  2:21         ` [PATCH v5 05/14] packfile-uri.txt: support for excluding commits and trees Teng Long
2021-08-25 23:52           ` Ævar Arnfjörð Bjarmason
2021-09-02 11:23             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 06/14] t5702: replace with "test_when_finished" for cleanup Teng Long
2021-08-25 23:55           ` Ævar Arnfjörð Bjarmason
2021-09-02 11:37             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 07/14] t5702: support for excluding commit objects Teng Long
2021-08-25  2:21         ` [PATCH v5 08/14] Add new parameter "carry_data" for "show_commit function Teng Long
2021-08-25  2:21         ` [PATCH v5 09/14] commit.h: add wrapped tags in commit struct Teng Long
2021-08-25 23:58           ` Ævar Arnfjörð Bjarmason
2021-09-02 12:17             ` Teng Long
2021-09-02 12:39           ` ZheNing Hu
2021-09-02 13:01             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 10/14] object.h: add referred tags in `referred_objects` struct Teng Long
2021-08-25  2:21         ` [PATCH v5 11/14] packfile-uri: support for excluding tag objects Teng Long
2021-08-25  2:21         ` [PATCH v5 12/14] packfile-uri.txt: " Teng Long
2021-08-25  2:21         ` [PATCH v5 13/14] t5702: add tag exclusion test case Teng Long
2021-08-25  2:21         ` [PATCH v5 14/14] pack-objects.c: introduce `want_exclude_object` function Teng Long
2021-10-19 11:38         ` [PATCH v6 00/12] packfile-uri: support excluding multiple object types Teng Long
2021-10-19 11:38           ` [PATCH v6 01/12] objects.c: introduce `exclude_level` enum Teng Long
2021-10-19 11:38           ` [PATCH v6 02/12] Introduce function `match_packfile_uri_exclusions` Teng Long
2021-10-19 11:38           ` [PATCH v6 03/12] Replace `show_data` with structure `show_info` Teng Long
2021-10-19 11:38           ` Teng Long [this message]
2021-10-19 11:38           ` [PATCH v6 05/12] t5702: test cases for `uploadpack.excludeobject` Teng Long
2021-10-19 11:38           ` [PATCH v6 06/12] packfile-uri: support for excluding commits Teng Long
2021-10-19 11:38           ` [PATCH v6 07/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` [PATCH v6 08/12] packfile-uri: support for excluding trees Teng Long
2021-10-19 11:38           ` [PATCH v6 09/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` [PATCH v6 10/12] packfile-uri: support for excluding tags Teng Long
2021-10-19 11:38           ` [PATCH v6 11/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` [PATCH v6 12/12] packfile-uri.txt: support multiple object types Teng Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b1d779b26a2f5d646b09131645b503d2e82a29ec.1634634814.git.tenglong@alibaba-inc.com \
    --to=dyroneteng@gmail.com \
    --cc=adlternative@gmail.com \
    --cc=avarab@gmail.com \
    --cc=bagasdotme@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).