From: Derrick Stolee <stolee@gmail.com>
To: Elijah Newren <newren@gmail.com>, git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Eric Wong" <e@80x24.org>,
"Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Lars Schneider" <larsxschneider@gmail.com>,
"Jonathan Nieder" <jrnieder@gmail.com>
Subject: Re: [RFC PATCH 2/5] t3427: accelerate this test by using fast-export and fast-import
Date: Mon, 26 Aug 2019 21:25:59 -0400 [thread overview]
Message-ID: <ad7446ac-34cc-b566-302b-053bf5dafc63@gmail.com> (raw)
In-Reply-To: <20190826235226.15386-3-newren@gmail.com>
On 8/26/2019 7:52 PM, Elijah Newren wrote:
> fast-export and fast-import can easily handle the simple rewrite that
> was being done by filter-branch, and should be significantly faster on
> systems with a slow fork. Timings from before and after on two laptops
> that I have access to (measured via `time ./t3427-rebase-subtree.sh`,
> i.e. including everything in this test -- not just the filter-branch or
> fast-export/fast-import pair):
>
> Linux: 4.305s -> 3.684s (~17% speedup)
> Mac: 10.128s -> 7.038s (~30% speedup)
Again, impressive speedup!
> Signed-off-by: Elijah Newren <newren@gmail.com>
> ---
> t/t3427-rebase-subtree.sh | 32 ++++++++++++++++++++++++--------
> 1 file changed, 24 insertions(+), 8 deletions(-)
>
> diff --git a/t/t3427-rebase-subtree.sh b/t/t3427-rebase-subtree.sh
> index d8640522a0..d05fcce5dc 100755
> --- a/t/t3427-rebase-subtree.sh
> +++ b/t/t3427-rebase-subtree.sh
> @@ -42,7 +42,9 @@ test_expect_failure REBASE_P \
> 'Rebase -Xsubtree --preserve-merges --onto commit 4' '
> reset_rebase &&
> git checkout -b rebase-preserve-merges-4 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
There's a lot of repetition in these changes. Is this a good time
to introduce a helper method? This trio of commands happens to be
difficult to parse, so I'd rather do it just once.
> git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
> verbose test "$(commit_message HEAD~)" = "files_subtree/master4"
> @@ -53,7 +55,9 @@ test_expect_failure REBASE_P \
> 'Rebase -Xsubtree --preserve-merges --onto commit 5' '
> reset_rebase &&
> git checkout -b rebase-preserve-merges-5 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --preserve-merges --onto files-master master &&
> verbose test "$(commit_message HEAD)" = "files_subtree/master5"
> @@ -64,7 +68,9 @@ test_expect_failure REBASE_P \
> 'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 4' '
> reset_rebase &&
> git checkout -b rebase-keep-empty-4 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
> verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
> @@ -75,7 +81,9 @@ test_expect_failure REBASE_P \
> 'Rebase -Xsubtree --keep-empty --preserve-merges --onto commit 5' '
> reset_rebase &&
> git checkout -b rebase-keep-empty-5 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
> verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
> @@ -86,7 +94,9 @@ test_expect_failure REBASE_P \
> 'Rebase -Xsubtree --keep-empty --preserve-merges --onto empty commit' '
> reset_rebase &&
> git checkout -b rebase-keep-empty-empty master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --keep-empty --preserve-merges --onto files-master master &&
> verbose test "$(commit_message HEAD)" = "Empty commit"
> @@ -96,7 +106,9 @@ test_expect_failure REBASE_P \
> test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
> reset_rebase &&
> git checkout -b rebase-onto-4 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --onto files-master master &&
> verbose test "$(commit_message HEAD~2)" = "files_subtree/master4"
> @@ -106,7 +118,9 @@ test_expect_failure 'Rebase -Xsubtree --onto commit 4' '
> test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
> reset_rebase &&
> git checkout -b rebase-onto-5 master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --onto files-master master &&
> verbose test "$(commit_message HEAD~)" = "files_subtree/master5"
> @@ -115,7 +129,9 @@ test_expect_failure 'Rebase -Xsubtree --onto commit 5' '
> test_expect_failure 'Rebase -Xsubtree --onto empty commit' '
> reset_rebase &&
> git checkout -b rebase-onto-empty master &&
> - git filter-branch --prune-empty -f --subdirectory-filter files_subtree &&
> + git fast-export --no-data HEAD -- files_subtree/ \
> + | sed -e "s%\([0-9a-f]\{40\} \)files_subtree/%\1%" \
> + | git fast-import --force --quiet &&
> git commit -m "Empty commit" --allow-empty &&
> git rebase -Xsubtree=files_subtree --onto files-master master &&
> verbose test "$(commit_message HEAD)" = "Empty commit"
>
next prev parent reply other threads:[~2019-08-27 1:26 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-22 18:26 RFC: Proposing git-filter-repo for inclusion in git.git Elijah Newren
2019-08-22 20:23 ` Junio C Hamano
2019-08-22 21:12 ` Elijah Newren
2019-08-22 21:34 ` Junio C Hamano
2019-08-26 23:52 ` [RFC PATCH 0/5] Remove git-filter-branch from git.git; host it elsewhere Elijah Newren
2019-08-26 23:52 ` [RFC PATCH 1/5] t6006: simplify and optimize empty message test Elijah Newren
2019-08-27 1:23 ` Derrick Stolee
2019-08-26 23:52 ` [RFC PATCH 2/5] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-27 1:25 ` Derrick Stolee [this message]
2019-08-26 23:52 ` [RFC PATCH 3/5] git-sh-i18n: work with external scripts Elijah Newren
2019-08-27 1:28 ` Derrick Stolee
2019-08-26 23:52 ` [RFC PATCH 4/5] Recommend git-filter-repo instead of git-filter-branch in documentation Elijah Newren
2019-08-27 1:32 ` Derrick Stolee
2019-08-27 6:23 ` Elijah Newren
2019-08-26 23:52 ` [RFC PATCH 5/5] Remove git-filter-branch, it is now external to git.git Elijah Newren
2019-08-27 1:39 ` [RFC PATCH 0/5] Remove git-filter-branch from git.git; host it elsewhere Derrick Stolee
2019-08-27 6:17 ` Elijah Newren
2019-08-27 7:03 ` Eric Wong
2019-08-27 8:43 ` Sergey Organov
2019-08-27 22:18 ` Elijah Newren
2019-08-28 8:52 ` Sergey Organov
2019-08-28 17:16 ` Elijah Newren
2019-08-28 19:03 ` Sergey Organov
2019-08-30 20:40 ` Johannes Schindelin
2019-08-30 23:22 ` Elijah Newren
2019-09-02 9:29 ` Johannes Schindelin
2019-09-03 17:37 ` Elijah Newren
2019-08-28 0:22 ` [PATCH v2 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-08-28 0:22 ` [PATCH v2 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-08-28 0:22 ` [PATCH v2 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-28 6:00 ` Eric Sunshine
2019-08-28 0:22 ` [PATCH v2 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-28 6:17 ` Eric Sunshine
2019-08-28 21:48 ` Elijah Newren
2019-08-28 0:22 ` [RFC PATCH v2 4/4] Remove git-filter-branch, it is now external to git.git Elijah Newren
2019-08-29 0:06 ` [PATCH v3 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-08-29 0:06 ` [PATCH v3 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-08-29 0:06 ` [PATCH v3 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-29 0:06 ` [PATCH v3 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-29 18:10 ` Eric Sunshine
2019-08-30 0:04 ` Elijah Newren
2019-08-29 0:06 ` [PATCH v3 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-08-30 5:57 ` [PATCH v4 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-08-30 5:57 ` [PATCH v4 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-09-02 14:47 ` Johannes Schindelin
2019-08-30 5:57 ` [PATCH v4 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-09-02 14:45 ` Johannes Schindelin
2019-08-30 5:57 ` [PATCH v4 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-30 5:57 ` [PATCH v4 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-09-03 18:55 ` [PATCH v5 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-09-03 18:55 ` [PATCH v5 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-09-03 21:08 ` Junio C Hamano
2019-09-03 21:58 ` Elijah Newren
2019-09-03 22:25 ` Junio C Hamano
2019-09-03 18:55 ` [PATCH v5 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-09-03 21:26 ` Junio C Hamano
2019-09-03 22:46 ` Junio C Hamano
2019-09-04 20:32 ` Elijah Newren
2019-09-03 18:55 ` [PATCH v5 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-09-03 21:40 ` Junio C Hamano
2019-09-04 20:30 ` Elijah Newren
2019-09-03 18:55 ` [PATCH v5 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-09-04 22:32 ` [PATCH v6 0/3] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-09-04 22:32 ` [PATCH v6 1/3] t6006: simplify, fix, and optimize empty message test Elijah Newren
2019-09-04 22:32 ` [PATCH v6 2/3] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-09-04 22:32 ` [PATCH v6 3/3] t9902: use a non-deprecated command for testing Elijah Newren
2019-08-23 3:00 ` RFC: Proposing git-filter-repo for inclusion in git.git Eric Wong
2019-08-23 18:06 ` Elijah Newren
2019-08-23 18:29 ` Elijah Newren
2019-08-28 11:09 ` Johannes Schindelin
2019-08-28 15:06 ` Junio C Hamano
2019-08-23 12:02 ` Derrick Stolee
2019-08-26 19:56 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ad7446ac-34cc-b566-302b-053bf5dafc63@gmail.com \
--to=stolee@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jrnieder@gmail.com \
--cc=larsxschneider@gmail.com \
--cc=newren@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).