git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Josh Smeaton <josh.smeaton@gmail.com>
To: git@vger.kernel.org
Subject: Bug: diff-index reports unrelated files as deleted when fsmonitor is enabled
Date: Sat, 4 Mar 2023 12:55:13 +1100	[thread overview]
Message-ID: <CAPbDM0d5Xe=KTLECDCJYtpzmBt2wqzZrMWVcbiisM+abopenig@mail.gmail.com> (raw)

git write-tree and then git diff-index reports thousands of files as `deleted`
when core.fsmonitor is true, even though the files are still present on the
file system.

What did you do before the bug happened? (Steps to reproduce your issue)

```
git config core.fsmonitor true
touch new-file.txt
git add new-file.txt
git diff-index --ignore-submodules --exit-code --no-color
--no-ext-diff $(git write-tree) --
```

What did you expect to happen? (Expected behavior)

I expected that no files would be reported as having changes.

What happened instead? (Actual behavior)

Thousands of unrelated files were reported as `deleted`.

```
git diff-index --ignore-submodules --exit-code --no-color
--no-ext-diff $(git write-tree) --
:100644 000000 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
0000000000000000000000000000000000000000 D src/tests/__init__.py
:100644 000000 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
0000000000000000000000000000000000000000 D src/tests/site/__init__.py
:100644 000000 d8edaad92120781998e811277f966551b12fc823
0000000000000000000000000000000000000000 D
src/tests/site/test_models.py

 ... thousands of other files
```

What's different between what you expected and what actually happened?

Thousands of unrelated files were reported as having been deleted.


Anything else you want to add:

Myself and a few co-workers all ran into this issue in the last few days. It
presented itself when pre-commit hooks were run (specifically with the
pre-commit
tool from github pre-commit).

When committing changes, the pre-commit tool runs the equivalent of:

`git diff-index --ignore-submodules --exit-code --no-color
--no-ext-diff $(git write-tree) --`

It then stashes and unstashes those changes, causing the files to be
removed from
the working tree.


```
$ git config core.fsmonitor true
$ git checkout -b test-branch
Switched to a new branch 'test-branch'

$ echo "test 3" >> newfile.txt
$ git add .
$ git status
On branch test-branch
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
new file:   newfile.txt

$ git commit -a -m "will it fail?"
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to
/Users/josh/.cache/pre-commit/patch1677888420-10129.
trim trailing whitespace.................................................Passed
check python ast.....................................(no files to check)Skipped
check for case conflicts.................................................Passed
[INFO] Restored changes from
/Users/josh/.cache/pre-commit/patch1677888420-10129.
[test-branch 7c899582c19] will it fail?
 1 file changed, 1 insertion(+)
 create mode 100644 newfile.txt

$ git status
On branch test-branch
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
deleted:    src/tests/__init__.py
deleted:    src/tests/site/__init__.py
deleted:    src/tests/site/test_models.py
        ... lots more files ...

no changes added to commit (use "git add" and/or "git commit -a")

$ git status | wc -l
   15298

$ git restore .

$ git status

On branch test-branch
nothing to commit, working tree clean
```

It can be reproduced, in our repo, without using pre-commit at all:

```

git diff-index --ignore-submodules --exit-code --no-color
--no-ext-diff $(git write-tree) --
:100644 000000 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
0000000000000000000000000000000000000000 D src/tests/__init__.py
:100644 000000 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
0000000000000000000000000000000000000000 D src/tests/site/__init__.py
:100644 000000 d8edaad92120781998e811277f966551b12fc823
0000000000000000000000000000000000000000 D
src/tests/site/test_models.py

 ... thousands of other files
```

Using `update-index` with the `--no-fsmonitor` flag prior to diff-index does the
correct thing:

```
$ git update-index --really-refresh --no-fsmonitor
warning: core.fsmonitor is set; remove it if you really want to
disable fsmonitor

$ git diff-index --ignore-submodules --exit-code --no-color
--no-ext-diff $(git write-tree) --
> no output
```

The problem persists when running `update-index` without disabling fsmonitor.

I have reported this issue to the pre-commit project, but it looks like an issue
with fsmonitor + write-tree.

See: pre-commit issues on github #2795



[System Info]
git version:
git version 2.39.2 (others reproduced with 2.39.1)
cpu: arm64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Darwin 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:38:37
PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6000 arm64
compiler info: clang: 14.0.0 (clang-1400.0.29.202)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/zsh


[Enabled Hooks]
pre-commit

             reply	other threads:[~2023-03-04  1:56 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-04  1:55 Josh Smeaton [this message]
2023-03-27  7:49 ` Bug: diff-index reports unrelated files as deleted when fsmonitor is enabled Johannes Schindelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAPbDM0d5Xe=KTLECDCJYtpzmBt2wqzZrMWVcbiisM+abopenig@mail.gmail.com' \
    --to=josh.smeaton@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).