How to delete files and directories from git commit history?

How to delete files and directories from git commit history?

[Steve Litt]

Hi all,

I have git 2.17.1 running on Void Linux 64 bit running the Linux
4.16.9_1 kernel, not available to the public in any way (yet). I have a
repository in my project's working directory, and push to a bare
repository on the hard disk.

My project (call it myproject) had a directory (call it docs/propdir)
that was unnecessary for the project, and I've decided I don't want to
offer the files in that directory as free software. So I need to delete
docs/propdir from all commits in the repository. I did the following,
while in my working repository's myproject directory:

git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD

After that command, I could git clone the working repo and then git
checkout to a commit stage that used to have the directory and files,
and they're not there. So far, so good.

But then I view all filenames from that directory that have ever been
in the project, as follows:

git cat-file --buffer --batch-all-objects \
 --batch-check='%(objecttype) %(objectname)' \
 | grep ^c | cut -d " "  -f 2 \
 | xargs -n 1 git ls-tree -r | sort | uniq \
 | grep propdir

The preceding command lists directory docs/propdir and all the files
it's ever contained. This makes me uneasy because if the filenames are
still there, I wonder if there's a route to get the files with a git
command. Second, I'd prefer that when my repo is exposed to the public,
people not know this directory and these files ever existed.

What command do I do to remove all mention of doc/propdir and its
files from my git history?

Thanks,

SteveT

Steve Litt 
June 2018 featured book: Twenty Eight Tales of Troubleshooting
http://www.troubleshooters.com/28

Re: How to delete files and directories from git commit history?

[Eckhard Maaß]

On Tue, Jun 12, 2018 at 03:44:13PM -0400, Steve Litt wrote:
> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD
...
> What command do I do to remove all mention of doc/propdir and its
> files from my git history?

Are you sure that you pruned all branches? I would have expected a
command like "git filter-branch ... -- --all" to prune every branch from
the directories.

Greetings,
Eckhard

Re: How to delete files and directories from git commit history?

[Christian Couder]

Hi,

On Tue, Jun 12, 2018 at 9:44 PM, Steve Litt <slitt@troubleshooters.com> wrote:

> My project (call it myproject) had a directory (call it docs/propdir)
> that was unnecessary for the project, and I've decided I don't want to
> offer the files in that directory as free software. So I need to delete
> docs/propdir from all commits in the repository. I did the following,
> while in my working repository's myproject directory:
>
> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD

[...]

> What command do I do to remove all mention of doc/propdir and its
> files from my git history?

Did you check the "CHECKLIST FOR SHRINKING A REPOSITORY" section of
the filter-branch man/help page?

Best,
Christian.

Re: How to delete files and directories from git commit history?

[Eric Sunshine]

On Tue, Jun 12, 2018 at 4:26 PM, Christian Couder
<christian.couder@gmail.com> wrote:
> On Tue, Jun 12, 2018 at 9:44 PM, Steve Litt <slitt@troubleshooters.com> wrote:
>
>> My project (call it myproject) had a directory (call it docs/propdir)
>> that was unnecessary for the project, and I've decided I don't want to
>> offer the files in that directory as free software. So I need to delete
>> docs/propdir from all commits in the repository. I did the following,
>> while in my working repository's myproject directory:
>>
>> git filter-branch --tree-filter 'rm -rf docs/propdir' HEAD
>
>> What command do I do to remove all mention of doc/propdir and its
>> files from my git history?
>
> Did you check the "CHECKLIST FOR SHRINKING A REPOSITORY" section of
> the filter-branch man/help page?

Also see BFG Repo Cleaner for an alternative.
https://rtyley.github.io/bfg-repo-cleaner/

Re: How to delete files and directories from git commit history?

[Junio C Hamano]

Steve Litt <slitt@troubleshooters.com> writes:

> But then I view all filenames from that directory that have ever been
> in the project, as follows:
>
> git cat-file --buffer --batch-all-objects \
>  --batch-check='%(objecttype) %(objectname)' \
>  | grep ^c | cut -d " "  -f 2 \
>  | xargs -n 1 git ls-tree -r | sort | uniq \
>  | grep propdir
>
> The preceding command lists directory docs/propdir and all the files
> it's ever contained....

In which repository did you run this?  As cat-file's documentation
clearly states, --batch-all-objects does not *care* about the
reachability, so even after rewriting the history using
filter-branch, as long as you have the original objects before your
history rewrite in the repository, it _will_ see those trees with
propdir, even if they are now unreachable thanks to your
filter-branch.  If you are doing the above in the original (or in a
local clone), try running "git repack -a -d && git prune
--expire=now" after filter-branch.