From: Elijah Newren <newren@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Derrick Stolee" <stolee@gmail.com>, "Eric Wong" <e@80x24.org>,
"Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Johannes Schindelin" <Johannes.Schindelin@gmx.de>,
"Lars Schneider" <larsxschneider@gmail.com>,
"Jonathan Nieder" <jrnieder@gmail.com>,
"Elijah Newren" <newren@gmail.com>
Subject: [PATCH v2 0/4] Warn about git-filter-branch usage and avoid it
Date: Tue, 27 Aug 2019 17:22:06 -0700 [thread overview]
Message-ID: <20190828002210.8862-1-newren@gmail.com> (raw)
In-Reply-To: <20190826235226.15386-1-newren@gmail.com>
Here's a series that shifts the focus slightly to warning about
git-filter-branch usage and avoiding it ourselves. I have retained
patch 4 but left it marked as RFC for further discussion. It appears
that folks generally seem to agree the first three patches are good
to include now -- assuming my small fixes correctly address their
requests and suggestions.
Changes since v1 (full range-diff below):
* I might have had a little fun with a thesaurus (just trying to give
reviewers something small to smile about...)
* addressed feedback from Eric and Stolee, as detailed below
* [Patch 2] factored out some common code
* [Patch 3] fixed links in asciidoc documentation to make them more
readable in both manpages and html format
* [Patch 3] added a warning blurb to git-filter-branch itself
In particular, it'd be helpful if people could take a look at the changes
to git-filter-branch.sh in patch 3 and comment on whether an environment
variable is fine or if we should make it a config setting or something.
Elijah Newren (4):
t6006: simplify and optimize empty message test
t3427: accelerate this test by using fast-export and fast-import
Recommend git-filter-repo instead of git-filter-branch
[RFC] Remove git-filter-branch, it is now external to git.git
.gitignore | 1 -
Documentation/git-fast-export.txt | 6 +-
Documentation/git-filter-branch.txt | 481 --------------------
Documentation/git-gc.txt | 17 +-
Documentation/git-rebase.txt | 2 +-
Documentation/git-replace.txt | 10 +-
Documentation/git-svn.txt | 4 +-
Documentation/githooks.txt | 7 +-
Makefile | 1 -
command-list.txt | 1 -
contrib/svn-fe/svn-fe.txt | 4 +-
git-filter-branch.sh | 662 ----------------------------
t/perf/p7000-filter-branch.sh | 24 -
t/t3427-rebase-subtree.sh | 22 +-
t/t6006-rev-list-format.sh | 5 +-
t/t7003-filter-branch.sh | 505 ---------------------
t/t7009-filter-branch-null-sha1.sh | 55 ---
t/t9902-completion.sh | 12 +-
18 files changed, 47 insertions(+), 1772 deletions(-)
delete mode 100644 Documentation/git-filter-branch.txt
delete mode 100755 git-filter-branch.sh
delete mode 100755 t/perf/p7000-filter-branch.sh
delete mode 100755 t/t7003-filter-branch.sh
delete mode 100755 t/t7009-filter-branch-null-sha1.sh
Range-diff:
1: 7ddbeea2ca = 1: 7ddbeea2ca t6006: simplify and optimize empty message test
2: 0172ca771e < -: ---------- t3427: accelerate this test by using fast-export and fast-import
3: b814cc7b65 < -: ---------- git-sh-i18n: work with external scripts
-: ---------- > 2: f18bd7a609 t3427: accelerate this test by using fast-export and fast-import
4: dcec36d113 ! 3: 7008c16984 Recommend git-filter-repo instead of git-filter-branch in documentation
@@ Metadata
Author: Elijah Newren <newren@gmail.com>
## Commit message ##
- Recommend git-filter-repo instead of git-filter-branch in documentation
+ Recommend git-filter-repo instead of git-filter-branch
- filter-branch suffers from a huge number of pitfalls that can result in
- incorrectly rewritten history, and many of the problems can easily go
- undetected until the new repository is in use. This can result in
- problems ranging from an even messier history than what led folks to
- filter-branch in the first place, to data loss or corruption. These
- issues cannot be backward compatibly fixed, so add a warning to the
- filter-branch manpage about this and recommand that another tool (such
- as filter-repo) be used instead.
+ filter-branch suffers from a deluge of disguised dangers that disfigure
+ history rewrites (i.e. deviate from the deliberate changes). Many of
+ these problems are unobtrusive and can easily go undiscovered until the
+ new repository is in use. This can result in problems ranging from an
+ even messier history than what led folks to filter-branch in the first
+ place, to data loss or corruption. These issues cannot be backward
+ compatibly fixed, so add a warning to both filter-branch and its manpage
+ recommending that another tool (such as filter-repo) be used instead.
Also, update other manpages that referenced filter-branch. Several of
these needed updates even if we could continue recommending
@@ Documentation/git-filter-branch.txt: SYNOPSIS
+WARNING
+-------
-+'git filter-branch' has a litany of gotchas that can and will cause
-+history to be rewritten incorrectly (in addition to abysmal
-+performance). These issues cannot be backward compatibly fixed and as
-+such, its use is not recommended. Please use an alternative history
-+filtering tool such as 'git filter-repo'. If you still need to use
-+'git filter-branch', please carefully read the "Safety" section of
-+https://public-inbox.org/git/CABPp-BEDOH-row-hxY4u_cP30ptqOpcCvPibwyZ2wBu142qUbA@mail.gmail.com/
-+and avoid as many of the pitfalls listed there as reasonably possible.
++'git filter-branch' has a plethora of pitfalls that can produce non-obvious
++manglings of the intended history rewrite (and can leave you with little
++time to investigate such problems since it has such abysmal performance).
++These safety and performance issues cannot be backward compatibly fixed and
++as such, its use is not recommended. Please use an alternative history
++filtering tool such as https://github.com/newren/git-filter-repo/[git
++filter-repo]. If you still need to use 'git filter-branch', please
++carefully read the "Safety" section of the message on the Git mailing list
++https://public-inbox.org/git/CABPp-BEDOH-row-hxY4u_cP30ptqOpcCvPibwyZ2wBu142qUbA@mail.gmail.com/[detailing
++the land mines of filter-branch] and vigilantly avoid as many of the
++hazards listed there as reasonably possible.
+
DESCRIPTION
-----------
@@ contrib/svn-fe/svn-fe.txt: The exit status does not reflect whether an error was
-git-svn(1), svn2git(1), svk(1), git-filter-branch(1), git-fast-import(1),
+git-svn(1), svn2git(1), svk(1), git-filter-repo(1), git-fast-import(1),
https://svn.apache.org/repos/asf/subversion/trunk/notes/dump-load-format.txt
+
+ ## git-filter-branch.sh (mode change 100755 => 100644) ##
+@@ git-filter-branch.sh: set_ident () {
+ finish_ident COMMITTER
+ }
+
++if [ -z "$FILTER_BRANCH_SQUELCH_WARNING" -a \
++ -z "$GIT_TEST_DISALLOW_ABBREVIATED_OPTIONS" ]; then
++ cat <<EOF
++WARNING: git-filter-branch has a glut of gotchas generating mangled history
++ rewrites. Please use an alternative filtering tool such as 'git
++ filter-repo' (https://github.com/newren/git-filter-repo/) instead.
++ See the filter-branch manual page for more details; to squelch
++ this warning and pause, set FILTER_BRANCH_SQUELCH_WARNING=1.
++
++EOF
++ sleep 5
++fi
++
+ USAGE="[--setup <command>] [--subdirectory-filter <directory>] [--env-filter <command>]
+ [--tree-filter <command>] [--index-filter <command>]
+ [--parent-filter <command>] [--msg-filter <command>]
5: 9dec8e06ee ! 4: ff3e04e558 Remove git-filter-branch, it is now external to git.git
@@ Metadata
## Commit message ##
Remove git-filter-branch, it is now external to git.git
+ git-filter-branch still exists, still has the same regression tests,
+ etc., but it is now being tracked in a separate repo that users will
+ need to download separately.
+
Signed-off-by: Elijah Newren <newren@gmail.com>
## .gitignore ##
@@ Documentation/git-filter-branch.txt (deleted)
-
-WARNING
--------
--'git filter-branch' has a litany of gotchas that can and will cause
--history to be rewritten incorrectly (in addition to abysmal
--performance). These issues cannot be backward compatibly fixed and as
--such, its use is not recommended. Please use an alternative history
--filtering tool such as 'git filter-repo'. If you still need to use
--'git filter-branch', please carefully read the "Safety" section of
--https://public-inbox.org/git/CABPp-BEDOH-row-hxY4u_cP30ptqOpcCvPibwyZ2wBu142qUbA@mail.gmail.com/
--and avoid as many of the pitfalls listed there as reasonably possible.
+-'git filter-branch' has a plethora of pitfalls that can produce non-obvious
+-manglings of the intended history rewrite (and can leave you with little
+-time to investigate such problems since it has such abysmal performance).
+-These safety and performance issues cannot be backward compatibly fixed and
+-as such, its use is not recommended. Please use an alternative history
+-filtering tool such as https://github.com/newren/git-filter-repo/[git
+-filter-repo]. If you still need to use 'git filter-branch', please
+-carefully read the "Safety" section of the message on the Git mailing list
+-https://public-inbox.org/git/CABPp-BEDOH-row-hxY4u_cP30ptqOpcCvPibwyZ2wBu142qUbA@mail.gmail.com/[detailing
+-the land mines of filter-branch] and vigilantly avoid as many of the
+-hazards listed there as reasonably possible.
-
-DESCRIPTION
------------
@@ git-filter-branch.sh (deleted)
- finish_ident COMMITTER
-}
-
+-if [ -z "$FILTER_BRANCH_SQUELCH_WARNING" -a \
+- -z "$GIT_TEST_DISALLOW_ABBREVIATED_OPTIONS" ]; then
+- cat <<EOF
+-WARNING: git-filter-branch has a glut of gotchas generating mangled history
+- rewrites. Please use an alternative filtering tool such as 'git
+- filter-repo' (https://github.com/newren/git-filter-repo/) instead.
+- See the filter-branch manual page for more details; to squelch
+- this warning and pause, set FILTER_BRANCH_SQUELCH_WARNING=1.
+-
+-EOF
+- sleep 5
+-fi
+-
-USAGE="[--setup <command>] [--subdirectory-filter <directory>] [--env-filter <command>]
- [--tree-filter <command>] [--index-filter <command>]
- [--parent-filter <command>] [--msg-filter <command>]
--
2.23.0.3.gcc10030edf.dirty
next prev parent reply other threads:[~2019-08-28 0:22 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-22 18:26 RFC: Proposing git-filter-repo for inclusion in git.git Elijah Newren
2019-08-22 20:23 ` Junio C Hamano
2019-08-22 21:12 ` Elijah Newren
2019-08-22 21:34 ` Junio C Hamano
2019-08-26 23:52 ` [RFC PATCH 0/5] Remove git-filter-branch from git.git; host it elsewhere Elijah Newren
2019-08-26 23:52 ` [RFC PATCH 1/5] t6006: simplify and optimize empty message test Elijah Newren
2019-08-27 1:23 ` Derrick Stolee
2019-08-26 23:52 ` [RFC PATCH 2/5] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-27 1:25 ` Derrick Stolee
2019-08-26 23:52 ` [RFC PATCH 3/5] git-sh-i18n: work with external scripts Elijah Newren
2019-08-27 1:28 ` Derrick Stolee
2019-08-26 23:52 ` [RFC PATCH 4/5] Recommend git-filter-repo instead of git-filter-branch in documentation Elijah Newren
2019-08-27 1:32 ` Derrick Stolee
2019-08-27 6:23 ` Elijah Newren
2019-08-26 23:52 ` [RFC PATCH 5/5] Remove git-filter-branch, it is now external to git.git Elijah Newren
2019-08-27 1:39 ` [RFC PATCH 0/5] Remove git-filter-branch from git.git; host it elsewhere Derrick Stolee
2019-08-27 6:17 ` Elijah Newren
2019-08-27 7:03 ` Eric Wong
2019-08-27 8:43 ` Sergey Organov
2019-08-27 22:18 ` Elijah Newren
2019-08-28 8:52 ` Sergey Organov
2019-08-28 17:16 ` Elijah Newren
2019-08-28 19:03 ` Sergey Organov
2019-08-30 20:40 ` Johannes Schindelin
2019-08-30 23:22 ` Elijah Newren
2019-09-02 9:29 ` Johannes Schindelin
2019-09-03 17:37 ` Elijah Newren
2019-08-28 0:22 ` Elijah Newren [this message]
2019-08-28 0:22 ` [PATCH v2 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-08-28 0:22 ` [PATCH v2 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-28 6:00 ` Eric Sunshine
2019-08-28 0:22 ` [PATCH v2 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-28 6:17 ` Eric Sunshine
2019-08-28 21:48 ` Elijah Newren
2019-08-28 0:22 ` [RFC PATCH v2 4/4] Remove git-filter-branch, it is now external to git.git Elijah Newren
2019-08-29 0:06 ` [PATCH v3 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-08-29 0:06 ` [PATCH v3 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-08-29 0:06 ` [PATCH v3 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-08-29 0:06 ` [PATCH v3 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-29 18:10 ` Eric Sunshine
2019-08-30 0:04 ` Elijah Newren
2019-08-29 0:06 ` [PATCH v3 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-08-30 5:57 ` [PATCH v4 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-08-30 5:57 ` [PATCH v4 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-09-02 14:47 ` Johannes Schindelin
2019-08-30 5:57 ` [PATCH v4 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-09-02 14:45 ` Johannes Schindelin
2019-08-30 5:57 ` [PATCH v4 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-08-30 5:57 ` [PATCH v4 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-09-03 18:55 ` [PATCH v5 0/4] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-09-03 18:55 ` [PATCH v5 1/4] t6006: simplify and optimize empty message test Elijah Newren
2019-09-03 21:08 ` Junio C Hamano
2019-09-03 21:58 ` Elijah Newren
2019-09-03 22:25 ` Junio C Hamano
2019-09-03 18:55 ` [PATCH v5 2/4] t3427: accelerate this test by using fast-export and fast-import Elijah Newren
2019-09-03 21:26 ` Junio C Hamano
2019-09-03 22:46 ` Junio C Hamano
2019-09-04 20:32 ` Elijah Newren
2019-09-03 18:55 ` [PATCH v5 3/4] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-09-03 21:40 ` Junio C Hamano
2019-09-04 20:30 ` Elijah Newren
2019-09-03 18:55 ` [PATCH v5 4/4] t9902: use a non-deprecated command for testing Elijah Newren
2019-09-04 22:32 ` [PATCH v6 0/3] Warn about git-filter-branch usage and avoid it Elijah Newren
2019-09-04 22:32 ` [PATCH v6 1/3] t6006: simplify, fix, and optimize empty message test Elijah Newren
2019-09-04 22:32 ` [PATCH v6 2/3] Recommend git-filter-repo instead of git-filter-branch Elijah Newren
2019-09-04 22:32 ` [PATCH v6 3/3] t9902: use a non-deprecated command for testing Elijah Newren
2019-08-23 3:00 ` RFC: Proposing git-filter-repo for inclusion in git.git Eric Wong
2019-08-23 18:06 ` Elijah Newren
2019-08-23 18:29 ` Elijah Newren
2019-08-28 11:09 ` Johannes Schindelin
2019-08-28 15:06 ` Junio C Hamano
2019-08-23 12:02 ` Derrick Stolee
2019-08-26 19:56 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190828002210.8862-1-newren@gmail.com \
--to=newren@gmail.com \
--cc=Johannes.Schindelin@gmx.de \
--cc=avarab@gmail.com \
--cc=e@80x24.org \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jrnieder@gmail.com \
--cc=larsxschneider@gmail.com \
--cc=peff@peff.net \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).