git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Teng Long <dyroneteng@gmail.com>
To: git@vger.kernel.org
Cc: gitster@pobox.com, avarab@gmail.com, jonathantanmy@google.com,
	bagasdotme@gmail.com, adlternative@gmail.com, stolee@gmail.com,
	Teng Long <tenglong@alibaba-inc.com>
Subject: [PATCH v6 12/12] packfile-uri.txt: support multiple object types
Date: Tue, 19 Oct 2021 19:38:36 +0800	[thread overview]
Message-ID: <79fc2c23cf053cb525d818b6b47e24c49b57f672.1634634814.git.tenglong@alibaba-inc.com> (raw)
In-Reply-To: <cover.1634634814.git.tenglong@alibaba-inc.com>

Signed-off-by: Teng Long <tenglong@alibaba-inc.com>
---
 Documentation/technical/packfile-uri.txt | 74 ++++++++++++++++++++----
 1 file changed, 63 insertions(+), 11 deletions(-)

diff --git a/Documentation/technical/packfile-uri.txt b/Documentation/technical/packfile-uri.txt
index f7eabc6c76..0e6d7c8dda 100644
--- a/Documentation/technical/packfile-uri.txt
+++ b/Documentation/technical/packfile-uri.txt
@@ -34,14 +34,69 @@ having it advertise `packfile-uris`, tolerating the client sending
 include some sort of non-trivial implementation in the Minimum Viable Product,
 at least so that we can test the client.
 
-This is the implementation: a feature, marked experimental, that allows the
-server to be configured by one or more `uploadpack.blobPackfileUri=<sha1>
-<uri>` entries. Whenever the list of objects to be sent is assembled, all such
-blobs are excluded, replaced with URIs. As noted in "Future work" below, the
-server can evolve in the future to support excluding other objects (or other
-implementations of servers could be made that support excluding other objects)
-without needing a protocol change, so clients should not expect that packfiles
-downloaded in this way only contain single blobs.
+This is the implementation: a feature, marked experimental, that allows
+the server to be configured by one or more entries with the format:
+
+    uploadpack.excludeobject=<object-hash> <level> <pack-hash> <uri>
+
+Value `<object-hash>` is the key of entry, and the object type can be
+blob, tree, commit, or tag. Value of entry has three parts,
+`<pack-hash>` is used to identify the packfile which contains the given
+`<object-hash>` object, and `<uri>` is the URI to download the packfile by
+client. For example, When a blob is configured with `uploadpack.excludeobject`
+that means whenever the blob to be send is assembled, the object will
+be excluded.
+
+In addition to excluding a single object like blob, sometimes it's
+hoped to exclude not only the object itself, but also all the related
+objects with it, like all the objects a tree contains or the ancestors
+that a commit can reach. In these cases, the `<level>` is designed to
+distinguish the scope of exclusion, it supports three levels:
+
+- Level 0: Excluding a single object itself, without any objects that
+  have a relationship with it. 
+
+- Level 1: Excluding object itself, and objects it contains.
+
+- Level 2: Excluding object itself, the objects it contains, and the
+  ancestors it can reach.
+
+If `<level>` is configured as 0, only the object itself will be
+excluded, no matter what the object type is. It is a common scenario
+for large size blobs, but it does much not sense for other object types
+(e.g. download a singe commit without downloading the blobs and tree
+in it).
+
+If `<level>` is configured as 1, not only the single object but also all
+the objects in it will be excluded. This applies to scenarios where
+it's wanted to exclude a specified non-blob object that includes some
+lage size objects.
+
+- If <object-hash> is a blob, the result is the same as level 0, because blob
+contains nothing just itself.
+
+- If <object-hash> is a tree, the tree itself, and all blobs and trees
+  in it will be excluded.
+
+- If <object-hash> is a commit, the commit itself, the referenced
+  root-tree, and all blobs and trees in the root-tree will be excluded.
+
+- If <object-hash> is a tag, the tag itself, the dereferenced commit
+  and all trees and blobs contained in its root-tree will be excluded.
+
+If `<level>` is configured as 2, not only the objects in the scope of
+level 1 , but also the reachable ancestors will be excluded if
+`<object-hash>` is commit or tag.
+
+Configuration compatibility
+---------------------------
+
+The old configuration of packfile-uri:
+
+    uploadpack.blobPackfileUri=<object-hash> <pack-hash> <uri>
+
+For the old configuration is compatible with the new one, but it only
+supports the exclusion of blob objects.
 
 Client design
 -------------
@@ -65,9 +120,6 @@ The protocol design allows some evolution of the server and client without any
 need for protocol changes, so only a small-scoped design is included here to
 form the MVP. For example, the following can be done:
 
- * On the server, more sophisticated means of excluding objects (e.g. by
-   specifying a commit to represent that commit and all objects that it
-   references).
  * On the client, resumption of clone. If a clone is interrupted, information
    could be recorded in the repository's config and a "clone-resume" command
    can resume the clone in progress. (Resumption of subsequent fetches is more
-- 
2.31.1.453.g945ddc3a74.dirty


      parent reply	other threads:[~2021-10-19 11:40 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-07  2:11 [PATCH] Packfile-uris support excluding commit objects Teng Long
2021-05-10 11:14 ` Ævar Arnfjörð Bjarmason
2021-05-18  8:49 ` [PATCH v2 0/3] packfile-uris: commit objects exclusion Teng Long
2021-05-18  8:49   ` [PATCH v2 1/3] packfile-uris: support for excluding commit object Teng Long
2021-05-19  4:28     ` Junio C Hamano
2021-05-20  4:46     ` Junio C Hamano
2021-05-18  8:49   ` [PATCH v2 2/3] packfile-uris.txt: " Teng Long
2021-05-18  8:49   ` [PATCH v2 3/3] t5702: excluding commits with packfile-uris Teng Long
2021-07-26  9:46   ` [PATCH v3 0/3] packfile-uris: commit objects exclusio Teng Long
2021-07-26  9:46     ` [PATCH v3 1/3] packfile-uris: support for excluding commit objects Teng Long
2021-07-26 18:15       ` Junio C Hamano
2021-07-26 19:45         ` Felipe Contreras
2021-08-11  1:44         ` Teng Long
2021-07-26  9:46     ` [PATCH v3 2/3] t5702: " Teng Long
2021-07-26 15:03       ` Ævar Arnfjörð Bjarmason
2021-08-11  1:46         ` [PATCH v3 1/3] packfile-uris: " Teng Long
2021-07-26  9:46     ` [PATCH v3 3/3] packfile-uri.txt: " Teng Long
2021-07-26 20:52       ` Junio C Hamano
2021-08-11  1:47         ` Teng Long
2021-07-26 12:34     ` [PATCH v3 0/3] packfile-uris: commit objects exclusio Ævar Arnfjörð Bjarmason
2021-08-11  1:48       ` Teng Long
2021-08-11  7:45     ` [PATCH v4 0/7] packfile-uris: commits and trees exclusion Teng Long
2021-08-11  7:45       ` [PATCH v4 1/7] pack-objects.c: introduce new method `match_packfile_uri_exclusions` Teng Long
2021-08-11  7:45       ` [PATCH v4 2/7] Add new parameter "carry_data" for "show_object" function Teng Long
2021-08-11  7:45       ` [PATCH v4 3/7] packfile-uri: support for excluding commit objects Teng Long
2021-08-11  7:45       ` [PATCH v4 4/7] packfile-uri: support for excluding tree objects Teng Long
2021-08-11  7:45       ` [PATCH v4 5/7] packfile-uri.txt: support for excluding commits and trees Teng Long
2021-08-11  9:59         ` Bagas Sanjaya
2021-08-11  7:45       ` [PATCH v4 6/7] t5702: replace with "test_when_finished" for cleanup Teng Long
2021-08-11  7:45       ` [PATCH v4 7/7] t5702: support for excluding commit objects Teng Long
2021-08-25  2:21       ` [PATCH v5 00/14] packfile-uris: commits, trees and tags exclusion Teng Long
2021-08-25  2:21         ` [PATCH v5 01/14] pack-objects.c: introduce new method `match_packfile_uri_exclusions` Teng Long
2021-08-25  2:21         ` [PATCH v5 02/14] Add new parameter "carry_data" for "show_object" function Teng Long
2021-08-26 20:45           ` Junio C Hamano
2021-09-02 11:08             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 03/14] packfile-uri: support for excluding commit objects Teng Long
2021-08-25 23:49           ` Ævar Arnfjörð Bjarmason
2021-09-02 12:26             ` Teng Long
2021-08-26 20:56           ` Junio C Hamano
2021-09-02 12:51             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 04/14] packfile-uri: support for excluding tree objects Teng Long
2021-08-25  2:21         ` [PATCH v5 05/14] packfile-uri.txt: support for excluding commits and trees Teng Long
2021-08-25 23:52           ` Ævar Arnfjörð Bjarmason
2021-09-02 11:23             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 06/14] t5702: replace with "test_when_finished" for cleanup Teng Long
2021-08-25 23:55           ` Ævar Arnfjörð Bjarmason
2021-09-02 11:37             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 07/14] t5702: support for excluding commit objects Teng Long
2021-08-25  2:21         ` [PATCH v5 08/14] Add new parameter "carry_data" for "show_commit function Teng Long
2021-08-25  2:21         ` [PATCH v5 09/14] commit.h: add wrapped tags in commit struct Teng Long
2021-08-25 23:58           ` Ævar Arnfjörð Bjarmason
2021-09-02 12:17             ` Teng Long
2021-09-02 12:39           ` ZheNing Hu
2021-09-02 13:01             ` Teng Long
2021-08-25  2:21         ` [PATCH v5 10/14] object.h: add referred tags in `referred_objects` struct Teng Long
2021-08-25  2:21         ` [PATCH v5 11/14] packfile-uri: support for excluding tag objects Teng Long
2021-08-25  2:21         ` [PATCH v5 12/14] packfile-uri.txt: " Teng Long
2021-08-25  2:21         ` [PATCH v5 13/14] t5702: add tag exclusion test case Teng Long
2021-08-25  2:21         ` [PATCH v5 14/14] pack-objects.c: introduce `want_exclude_object` function Teng Long
2021-10-19 11:38         ` [PATCH v6 00/12] packfile-uri: support excluding multiple object types Teng Long
2021-10-19 11:38           ` [PATCH v6 01/12] objects.c: introduce `exclude_level` enum Teng Long
2021-10-19 11:38           ` [PATCH v6 02/12] Introduce function `match_packfile_uri_exclusions` Teng Long
2021-10-19 11:38           ` [PATCH v6 03/12] Replace `show_data` with structure `show_info` Teng Long
2021-10-19 11:38           ` [PATCH v6 04/12] Introduce `uploadpack.excludeobject` configuration Teng Long
2021-10-19 11:38           ` [PATCH v6 05/12] t5702: test cases for `uploadpack.excludeobject` Teng Long
2021-10-19 11:38           ` [PATCH v6 06/12] packfile-uri: support for excluding commits Teng Long
2021-10-19 11:38           ` [PATCH v6 07/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` [PATCH v6 08/12] packfile-uri: support for excluding trees Teng Long
2021-10-19 11:38           ` [PATCH v6 09/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` [PATCH v6 10/12] packfile-uri: support for excluding tags Teng Long
2021-10-19 11:38           ` [PATCH v6 11/12] t5702: test cases " Teng Long
2021-10-19 11:38           ` Teng Long [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=79fc2c23cf053cb525d818b6b47e24c49b57f672.1634634814.git.tenglong@alibaba-inc.com \
    --to=dyroneteng@gmail.com \
    --cc=adlternative@gmail.com \
    --cc=avarab@gmail.com \
    --cc=bagasdotme@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jonathantanmy@google.com \
    --cc=stolee@gmail.com \
    --cc=tenglong@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).