git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Dhruva Krishnamurthy <dhruvakm@gmail.com>
To: git@vger.kernel.org
Subject: Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42
Date: Tue, 30 Apr 2024 22:26:32 -0700	[thread overview]
Message-ID: <CAKOHPAn1btewYTdLYWpW+fOaXMY+JQZsLCQxUSwoUqnnFN_ohA@mail.gmail.com> (raw)

Hello,
Cloning by specifying depth exhibits performance regression in
pack-objects (~20x). The repository I am cloning is on NFS (mounted
with NFSv3 & positive lookup cache enabled).

Ran under 'perf' command to capture profiling information to see if
something really stands out. There is a significant overhead in calls
to file open/open64, fstat64 & mmap/munmap in git 2.44 compared to git
2.42. Not sure if there is an increase in the number of calls or
something more is done.

Could someone please guide me on how to troubleshoot this better?

--- Details of the test environment and the clone commands with output ---
# There are 10 loose objects under objects/08
$ git count-objects -vH
count: 3627
size: 25.84 MiB
in-pack: 1108374
packs: 2
size-pack: 303.38 MiB
prune-packable: 0
garbage: 0
size-garbage: 0 bytes

# Simple driver script to enable performance tracking for upload-pack only
$ cat trace-git-upload-pack
#!/usr/bin/env bash
export GIT_TRACE_PERFORMANCE=true
exec git-upload-pack "$@"

# git clone with 2.42: pack objects take 17s
$ /opt/git/bin/git clone --no-checkout --no-local --depth=500
--upload-pack=$(pwd)/trace-git-upload-pack .. prod
Cloning into 'prod'...
remote: Enumerating objects: 669941, done.
remote: Counting objects: 100% (669941/669941), done.
remote: Compressing objects: 100% (154988/154988), done.
Receiving objects: 100% (669941/669941), 144.54 MiB | 25.78 MiB/s, done.
remote: Total 669941 (delta 533745), reused 645666 (delta 512193), pack-reused 0
remote: 05:35:40.654828 trace.c:414             performance:
17.098198597 s: git command: git --shallow-file '' pack-objects --revs
--thin --stdout --shallow --progress --delta-base-offset --include-tag
        05:35:40.708162 trace.c:414             performance:
24.764812264 s: git command: git-upload-pack /large_repo/perf/..
Resolving deltas: 100% (533745/533745), done.
Checking connectivity: 669940, done.

# git clone with 2.44: pack objects take 325s
$ /opt/gitn/bin/git clone --no-checkout --no-local --depth=500
--upload-pack=$(pwd)/trace-git-upload-pack .. dev
Cloning into 'dev'...
remote: Enumerating objects: 669941, done.
remote: Counting objects: 100% (669941/669941), done.
remote: Compressing objects: 100% (154988/154988), done.
Receiving objects: 100% (669941/669941), 144.66 MiB | 29.08 MiB/s, done.
remote: Total 669941 (delta 533742), reused 645666 (delta 512193),
pack-reused 0 (from 0)
remote: 05:42:01.017156 trace.c:414             performance:
325.552424902 s: git command: git --shallow-file '' pack-objects
--revs --thin --stdout --shallow --progress --delta-base-offset
--include-tag
        05:42:01.063013 trace.c:414             performance:
330.965731114 s: git command: git-upload-pack /large_repo/perf/..
Resolving deltas: 100% (533742/533742), done.
Checking connectivity: 669940, done.

Best regards,
Dhruva


             reply	other threads:[~2024-05-01  5:27 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-05-01  5:26 Dhruva Krishnamurthy [this message]
2024-05-01 22:00 ` using tree as attribute source is slow, was Re: Help troubleshoot performance regression cloning with depth: git 2.44 vs git 2.42 Jeff King
2024-05-01 22:37   ` rsbecker
2024-05-01 22:40   ` Junio C Hamano
2024-05-02  0:33   ` Taylor Blau
2024-05-02 17:33     ` Taylor Blau
2024-05-02 17:44       ` Junio C Hamano
2024-05-02 17:55         ` Taylor Blau
2024-05-02 19:01           ` Karthik Nayak
2024-05-02 21:08             ` Junio C Hamano
2024-05-03  5:37               ` Dhruva Krishnamurthy
2024-05-03 15:34                 ` Re* " Junio C Hamano
2024-05-03 17:46                   ` Jeff King
2024-05-06 20:28                     ` Taylor Blau
2024-05-13 20:16                   ` John Cai
2024-05-02 18:34         ` Dhruva Krishnamurthy
2024-05-02  0:45   ` Dhruva Krishnamurthy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAKOHPAn1btewYTdLYWpW+fOaXMY+JQZsLCQxUSwoUqnnFN_ohA@mail.gmail.com \
    --to=dhruvakm@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).