git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Andrew Wansink <wansink@uber.com>
To: git@vger.kernel.org
Subject: Subject: [RFC PATCH] upload_pack.c: make deepen-not more tree-ish
Date: Fri, 10 Feb 2023 13:31:54 -0800	[thread overview]
Message-ID: <CA+tAvojz0u7AbcNnY1qyy3VznKhYTiAO1dL+rfOD3O6mOtsa8A@mail.gmail.com> (raw)

This unlocks `git clone --shallow-exclude=<commit-sha1>`

git-clone only accepts --shallow-excude arguments where
the argument is a branch or tag because upload_pack only
searches deepen-not arguments for branches and tags.

Make process_deepen_not search for commit objects if no
branch or tag is found then add them to the deepen_not
list.

Signed-off-by: Andrew Wansink <wansink@uber.com>
---

At Uber we have a lot of patches in CI simultaneously,
the CI jobs will frequently clone the monorepo multiple
times for each patch.  They do this to calculate diffs
between a patch and its parent commit.

One optimisation in this flow is to clone only to a specific
depth, this may or may not work, depending on how old the
patch is.  In this case we have to --unshallow or discard
the shallow clone and fully clone the repo.

This patch would allow us to clone to exactly the depth we
need to find a patch's parent commit.

 t/t5500-fetch-pack.sh | 30 ++++++++++++++++++++++++++++++
 upload-pack.c         | 35 +++++++++++++++++++++++++++++++----
 2 files changed, 61 insertions(+), 4 deletions(-)

diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index d18f2823d86..8d5045cc1b9 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -899,6 +899,36 @@ test_expect_success 'shallow clone exclude tag two' '
  )
 '

+test_expect_success 'shallow clone exclude commit' '
+ test_create_repo shallow-exclude-commit &&
+ (
+ cd shallow-exclude-commit &&
+ test_commit one &&
+ test_commit two &&
+ test_commit three &&
+ commit_two_sha1=$(git log -n 1 --pretty=tformat:%h HEAD^) &&
+ git clone --shallow-exclude=${commit_two_sha1} "file://$(pwd)/."
../shallow3-by-commit &&
+ git -C ../shallow3-by-commit log --pretty=tformat:%s HEAD >actual &&
+ git log -n 1 --pretty=tformat:%s HEAD >expected &&
+ test_cmp expected actual
+ )
+'
+
+test_expect_success 'shallow clone exclude commit^' '
+ test_create_repo shallow-exclude-commit-carat &&
+ (
+ cd shallow-exclude-commit-carat &&
+ test_commit one &&
+ test_commit two &&
+ test_commit three &&
+ commit_two_sha1=$(git log -n 1 --pretty=tformat:%h HEAD^) &&
+ git clone --shallow-exclude=${commit_two_sha1}^ "file://$(pwd)/."
../shallow23-by-commit &&
+ git -C ../shallow23-by-commit log --pretty=tformat:%s HEAD >actual &&
+ git log -n 2 --pretty=tformat:%s HEAD >expected &&
+ test_cmp expected actual
+ )
+'
+
 test_expect_success 'fetch exclude tag one' '
  git -C shallow12 fetch --shallow-exclude one origin &&
  git -C shallow12 log --pretty=tformat:%s origin/main >actual &&
diff --git a/upload-pack.c b/upload-pack.c
index 551f22ffa5d..0c8594f4744 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -985,10 +985,37 @@ static int process_deepen_not(const char *line,
struct string_list *deepen_not,
  if (skip_prefix(line, "deepen-not ", &arg)) {
  char *ref = NULL;
  struct object_id oid;
- if (expand_ref(the_repository, arg, strlen(arg), &oid, &ref) != 1)
- die("git upload-pack: ambiguous deepen-not: %s", line);
- string_list_append(deepen_not, ref);
- free(ref);
+
+ switch (expand_ref(the_repository, arg, strlen(arg), &oid, &ref)) {
+ case 1:
+ // tag or branch matching arg found
+ string_list_append(deepen_not, ref);
+ free(ref);
+ break;
+ case 0: {
+ // no tags or branches matching arg
+ struct object *obj = NULL;
+ struct commit *commit = NULL;
+
+ if (get_oid(arg, &oid))
+ die("git upload-pack: deepen-not: no ref or object %s", arg);
+
+ obj = parse_object(the_repository, &oid);
+ if (!obj)
+ die("git upload-pack: deepen-not: object could not be parsed: %s", arg);
+
+ commit = (struct commit *)peel_to_type(arg, 0, obj, OBJ_COMMIT);
+ if (!commit)
+ die("git upload-pack: deepen-not: object not a commit: %s", arg);
+
+ string_list_append(deepen_not, oid_to_hex(&commit->object.oid));
+ break;
+ }
+ default:
+ // more than 1 tag or branch matches arg
+ die("git upload-pack: ambiguous deepen-not: %s", arg);
+ }
+
  *deepen_rev_list = 1;
  return 1;
  }
-- 
2.39.1

             reply	other threads:[~2023-02-10 21:32 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-10 21:31 Andrew Wansink [this message]
2023-02-11 22:23 ` [RFC PATCH] upload_pack.c: make deepen-not more tree-ish Andrew Wansink
2023-02-11 22:40   ` Andrew Wansink
     [not found]   ` <CAL3xRKdCkAAR0r3jyKFy+TtUi65LQcHaste=2WCqYHtwi8cUhw@mail.gmail.com>
2023-02-12 14:12     ` Son Luong Ngoc

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+tAvojz0u7AbcNnY1qyy3VznKhYTiAO1dL+rfOD3O6mOtsa8A@mail.gmail.com \
    --to=wansink@uber.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).