git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com>
To: git@vger.kernel.org
Cc: newren@gmail.com, stolee@gmail.com, gitster@pobox.com,
	Derrick Stolee <derrickstolee@github.com>
Subject: [PATCH 0/6] Sparse Index: Integrate with merge, cherry-pick, rebase, and revert
Date: Tue, 17 Aug 2021 17:08:38 +0000	[thread overview]
Message-ID: <pull.1019.git.1629220124.gitgitgadget@gmail.com> (raw)

This series integrates the sparse index with commands that perform merges
such as 'git merge', 'git cherry-pick', 'git revert' (free with
cherry-pick), and 'git rebase'.

When the ORT merge strategy is enabled, this allows most merges to succeed
without expanding the sparse index, leading to significant performance
gains. I tested these changes against an internal monorepo with over 2
million paths at HEAD but with a sparse-checkout that only has ~60,000 files
within the sparse-checkout cone. 'git merge' commands went from 5-6 seconds
to 0.750-1.250s.

In the case of the recursive merge strategy, the sparse index is expanded
before the recursive algorithm proceeds. We expect that this is as good as
we can get with that strategy. When the strategy shifts to ORT as the
default, then this will not be a problem except for users who decide to
change the behavior.

Most of the hard work was done by previous series, such as
ds/sparse-index-ignored-files (which this series is based on).

Thanks, -Stolee

Derrick Stolee (6):
  t1092: use ORT merge strategy
  diff: ignore sparse paths in diffstat
  merge: make sparse-aware with ORT
  merge-ort: expand only for out-of-cone conflicts
  t1092: add cherry-pick, rebase tests
  sparse-index: integrate with cherry-pick and rebase

 builtin/merge.c                          |  3 +
 builtin/rebase.c                         |  6 ++
 builtin/revert.c                         |  3 +
 diff.c                                   |  8 ++
 merge-ort.c                              | 16 ++++
 merge-recursive.c                        |  3 +
 t/t1092-sparse-checkout-compatibility.sh | 97 +++++++++++++++++++++---
 7 files changed, 124 insertions(+), 12 deletions(-)


base-commit: febef675f051eb08896751bb5661b6deb5579ead
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1019%2Fderrickstolee%2Fsparse-index%2Fmerge-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1019/derrickstolee/sparse-index/merge-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1019
-- 
gitgitgadget

             reply	other threads:[~2021-08-17 17:09 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-17 17:08 Derrick Stolee via GitGitGadget [this message]
2021-08-17 17:08 ` [PATCH 1/6] t1092: use ORT merge strategy Derrick Stolee via GitGitGadget
2021-08-18 17:16   ` Taylor Blau
2021-08-18 18:10   ` Junio C Hamano
2021-08-18 18:42     ` Derrick Stolee
2021-08-18 22:22       ` Junio C Hamano
2021-08-20 21:23       ` Elijah Newren
2021-08-20 23:32         ` Junio C Hamano
2021-08-21  0:20           ` Elijah Newren
2021-08-21  4:20             ` Junio C Hamano
2021-08-21 23:48               ` Elijah Newren
2021-08-17 17:08 ` [PATCH 2/6] diff: ignore sparse paths in diffstat Derrick Stolee via GitGitGadget
2021-08-20 21:32   ` Elijah Newren
2021-08-24 18:30     ` Derrick Stolee
2021-08-27 22:27       ` Elijah Newren
2021-08-17 17:08 ` [PATCH 3/6] merge: make sparse-aware with ORT Derrick Stolee via GitGitGadget
2021-08-20 21:40   ` Elijah Newren
2021-08-17 17:08 ` [PATCH 4/6] merge-ort: expand only for out-of-cone conflicts Derrick Stolee via GitGitGadget
2021-08-20 21:53   ` Elijah Newren
2021-08-17 17:08 ` [PATCH 5/6] t1092: add cherry-pick, rebase tests Derrick Stolee via GitGitGadget
2021-08-20 21:58   ` Elijah Newren
2021-08-17 17:08 ` [PATCH 6/6] sparse-index: integrate with cherry-pick and rebase Derrick Stolee via GitGitGadget
2021-08-21  0:12   ` Elijah Newren
2021-08-24 21:52 ` [PATCH v2 0/6] Sparse Index: Integrate with merge, cherry-pick, rebase, and revert Derrick Stolee via GitGitGadget
2021-08-24 21:52   ` [PATCH v2 1/6] diff: ignore sparse paths in diffstat Derrick Stolee via GitGitGadget
2021-08-24 21:52   ` [PATCH v2 2/6] merge: make sparse-aware with ORT Derrick Stolee via GitGitGadget
2021-08-27 22:43     ` Elijah Newren
2021-08-30 17:18       ` Derrick Stolee
2021-09-08  1:49         ` Derrick Stolee
2021-08-24 21:52   ` [PATCH v2 3/6] merge-ort: expand only for out-of-cone conflicts Derrick Stolee via GitGitGadget
2021-08-27 22:47     ` Elijah Newren
2021-08-30 17:21       ` Derrick Stolee
2021-08-24 21:52   ` [PATCH v2 4/6] t1092: add cherry-pick, rebase tests Derrick Stolee via GitGitGadget
2021-08-24 21:52   ` [PATCH v2 5/6] sequencer: ensure full index if not ORT strategy Derrick Stolee via GitGitGadget
2021-08-24 21:52   ` [PATCH v2 6/6] sparse-index: integrate with cherry-pick and rebase Derrick Stolee via GitGitGadget
2021-08-27 22:56   ` [PATCH v2 0/6] Sparse Index: Integrate with merge, cherry-pick, rebase, and revert Elijah Newren
2021-08-30 17:26     ` Derrick Stolee
2021-09-08 11:23   ` [PATCH v3 " Derrick Stolee via GitGitGadget
2021-09-08 11:23     ` [PATCH v3 1/6] diff: ignore sparse paths in diffstat Derrick Stolee via GitGitGadget
2021-09-08 11:23     ` [PATCH v3 2/6] merge: make sparse-aware with ORT Derrick Stolee via GitGitGadget
2021-09-08 11:23     ` [PATCH v3 3/6] merge-ort: expand only for out-of-cone conflicts Derrick Stolee via GitGitGadget
2021-09-08 11:23     ` [PATCH v3 4/6] t1092: add cherry-pick, rebase tests Derrick Stolee via GitGitGadget
2021-09-08 11:24     ` [PATCH v3 5/6] sequencer: ensure full index if not ORT strategy Derrick Stolee via GitGitGadget
2021-09-08 11:24     ` [PATCH v3 6/6] sparse-index: integrate with cherry-pick and rebase Derrick Stolee via GitGitGadget
2021-09-09 14:16     ` [PATCH v3 0/6] Sparse Index: Integrate with merge, cherry-pick, rebase, and revert Elijah Newren

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=pull.1019.git.1629220124.gitgitgadget@gmail.com \
    --to=gitgitgadget@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=newren@gmail.com \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).