Git bug report - disk errors on Windows after push

Git bug report - disk errors on Windows after push

[Attila Csosz]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)
Pushing to a directory directly.

What did you expect to happen? (Expected behavior)
Good behavior

What happened instead? (Actual behavior)
Disk errors. After pushing to a directory (my origin field is C:\Work 
for example) the git repository will be corrupted.
Moreover causing disk errors not only in the target git bare repository. 
I've loosed some other files.
Cannot be reproduced exactly. However I have decided to report this 
problem.
I've experienced this problem for several years.

What's different between what you expected and what actually happened?
Described.

Anything else you want to add:

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.31.1.windows.1
cpu: x86_64
built from commit: c5f0be26a7e3846e3b6268d1c6c4800d838c6bbb
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Windows 10.0 19044
compiler info: gnuc: 10.2
libc info: no libc information available
$SHELL (typically, interactive shell): C:\Program Files\Git\usr\bin\bash.exe


[Enabled Hooks]

Re: Git bug report - disk errors on Windows after push

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]

Hey,

On 2022-04-18 at 07:47:07, Attila Csosz wrote:
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> Pushing to a directory directly.
> 
> What did you expect to happen? (Expected behavior)
> Good behavior
> 
> What happened instead? (Actual behavior)
> Disk errors. After pushing to a directory (my origin field is C:\Work for
> example) the git repository will be corrupted.
> Moreover causing disk errors not only in the target git bare repository.
> I've loosed some other files.
> Cannot be reproduced exactly. However I have decided to report this problem.
> I've experienced this problem for several years.

There's nothing that Git should be able to do as a normal operating
system that causes disk errors.  If you're seeing those kinds of errors,
the likelihood is that your disk is bad and you need to replace it.

Do note that if you're storing your data in a directory managed by a
cloud syncing service (e.g., Dropbox or OneDrive), then it is known that
those file systems corrupt Git repositories and you shouldn't use them
with Git.  That's due to the way they sync data.

However, if this is a regular file system (e.g., NTFS), then Git
shouldn't be able to cause disk problems, and if it could, that would be
a serious defect in Windows, so it's almost certainly going to be a
hardware problem.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Git bug report - disk errors on Windows after push

[Attila Csosz]

I've experienced the problem at least 3 different external disk.
I'm not using cloud syncing service (e.g., Dropbox or OneDrive) for git.

Attila

On 4/18/2022 19:37, brian m. carlson wrote:
> Hey,
>
> On 2022-04-18 at 07:47:07, Attila Csosz wrote:
>> Thank you for filling out a Git bug report!
>> Please answer the following questions to help us understand your issue.
>>
>> What did you do before the bug happened? (Steps to reproduce your issue)
>> Pushing to a directory directly.
>>
>> What did you expect to happen? (Expected behavior)
>> Good behavior
>>
>> What happened instead? (Actual behavior)
>> Disk errors. After pushing to a directory (my origin field is C:\Work for
>> example) the git repository will be corrupted.
>> Moreover causing disk errors not only in the target git bare repository.
>> I've loosed some other files.
>> Cannot be reproduced exactly. However I have decided to report this problem.
>> I've experienced this problem for several years.
> There's nothing that Git should be able to do as a normal operating
> system that causes disk errors.  If you're seeing those kinds of errors,
> the likelihood is that your disk is bad and you need to replace it.
>
> Do note that if you're storing your data in a directory managed by a
> cloud syncing service (e.g., Dropbox or OneDrive), then it is known that
> those file systems corrupt Git repositories and you shouldn't use them
> with Git.  That's due to the way they sync data.
>
> However, if this is a regular file system (e.g., NTFS), then Git
> shouldn't be able to cause disk problems, and if it could, that would be
> a serious defect in Windows, so it's almost certainly going to be a
> hardware problem.

Re: Git bug report - disk errors on Windows after push

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1443 bytes --]

[Please avoid top-posting on this list.]

On 2022-04-19 at 03:52:23, Attila Csosz wrote:
> I've experienced the problem at least 3 different external disk.
> I'm not using cloud syncing service (e.g., Dropbox or OneDrive) for git.

I don't doubt that's the case.  However, it's not possible for Git to
corrupt a disk just by writing to it.  Git, as a normal unprivileged
program, can only open files and perform normal read and write
operations on them.  Management of the file system, including integrity,
is the responsibility of the operating system.  If running Git on
Windows causes files outside of the repository to be corrupted, that's
either a hardware problem or a bug in Windows.

If you're seeing this problem, it could be that you have a series of bad
disks (say, if you bought a set of cheap flash drives), that the drive
isn't being removed properly, that you have some sort of broken driver
or malware, that there's some other hardware problem (e.g., a bad dock,
USB device, CPU, or memory), or just that there's a previously unknown
bug in Windows.

None of this is Git's fault, and it's really up to you to figure out
which of these it is.  You might try isolating it by using a different
computer to write to the repository first, to see if something about
your current machine is the problem, and then try isolating other
causes.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: Git bug report - disk errors on Windows after push

[Neeraj Singh]

On 4/18/2022 8:52 PM, Attila Csosz wrote:
> I've experienced the problem at least 3 different external disk.
> I'm not using cloud syncing service (e.g., Dropbox or OneDrive) for git.
> 
> Attila
> 
Hi Attila,
Could you please share a bit more information about the disks that are 
being corrupted and the specific error messages you're seeing?

What is the underlying storage type (e.g. USB key, external hard drive,
etc)?  Also, which filesystem is the drive formatted as?

Could you please save one instance of a volume in the corrupted state so 
that we can consider further investigations?

I believe the output of "fsutil fsinfo volumeinfo <drive letter>" and
"fsutil fsinfo drivetype <drive letter>" should give us some useful info 
about the volume that's being corrupted.

Thanks,
Neeraj Singh,
Windows FileSystem Dev

Re: Git bug report - disk errors on Windows after push

[Attila Csosz]

Please check this link also
https://github.community/t/what-could-be-the-reason-my-git-repositories-keep-getting-corrupted-broken/184711

You can found many link on web on 'git corrupted repository'. In my
opinion when a git repository is corrupted on Windows there is a good
chance to HDD errors.

I suggest also run intensive test on Windows (run a bat file commiting
and pushing to a bare directory type repository). Check your HDD
healthy status after 10-20 run of 'git push'.

My git software is come form 'git-scm.com' and I'm using TortoiseGit.
My git software is up to date.
Last time my laptop HDD is crashed loosing sensitive datas. I will
never push to a local repo I'm using online repo-s like github.

///
fsutil fsinfo volumeinfo e:
Volume Name : Seagate Backup Plus Drive
Volume Serial Number : <removed by hand>
Max Component Length : 255
File System Name : NTFS
Is ReadWrite
Not Thinly-Provisioned
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's
Supports file-based Compression
Supports Disk Quotas
Supports Sparse files
Supports Reparse Points
Returns Handle Close Result Information
Supports POSIX-style Unlink and Rename
Supports Object Identifiers
Supports Encrypted File System
Supports Named Streams
Supports Transactions
Supports Hard Links
Supports Extended Attributes
Supports Open By FileID
Supports USN Journal
///

Maybe this email not reaching the main mailing list I'm somehow blacklisted.


Attila

On Thu, Apr 21, 2022 at 12:54 AM Neeraj Singh <nksingh85@gmail.com> wrote:
>
> On 4/18/2022 8:52 PM, Attila Csosz wrote:
> > I've experienced the problem at least 3 different external disk.
> > I'm not using cloud syncing service (e.g., Dropbox or OneDrive) for git.
> >
> > Attila
> >
> Hi Attila,
> Could you please share a bit more information about the disks that are
> being corrupted and the specific error messages you're seeing?
>
> What is the underlying storage type (e.g. USB key, external hard drive,
> etc)?  Also, which filesystem is the drive formatted as?
>
> Could you please save one instance of a volume in the corrupted state so
> that we can consider further investigations?
>
> I believe the output of "fsutil fsinfo volumeinfo <drive letter>" and
> "fsutil fsinfo drivetype <drive letter>" should give us some useful info
> about the volume that's being corrupted.
>
> Thanks,
> Neeraj Singh,
> Windows FileSystem Dev

Re: Git bug report - disk errors on Windows after push

[Neeraj Singh]

On 4/20/2022 7:29 PM, Attila Csosz wrote:
> Please check this link also
> https://github.community/t/what-could-be-the-reason-my-git-repositories-keep-getting-corrupted-broken/184711
> 
> You can found many link on web on 'git corrupted repository'. In my
> opinion when a git repository is corrupted on Windows there is a good
> chance to HDD errors.
> 
> I suggest also run intensive test on Windows (run a bat file commiting
> and pushing to a bare directory type repository). Check your HDD
> healthy status after 10-20 run of 'git push'.
> 
> My git software is come form 'git-scm.com' and I'm using TortoiseGit.
> My git software is up to date.
> Last time my laptop HDD is crashed loosing sensitive datas. I will
> never push to a local repo I'm using online repo-s like github.
> 

Do you currently have a drive in the bad state? Is write-caching enabled 
for the drive 
(https://docs.microsoft.com/en-us/troubleshoot/windows-server/backup-and-storage/turn-disk-write-caching-on-off)?

Git hammers the filesystem in a way that's not typically done for a USB 
drive (especially issuing some "FlushFileBuffers" calls).  So it 
possibly exposes some inherent flakiness in your setup. For instance, 
your drives might be good, but something could be wrong with your USB 
cable or the power supply.

If you have a drive in the bad state, a "chkdsk /scan /X <drive letter>" 
might give some hints about what specifically is wrong with that filesystem.

I'd recommend preferring a cloud-based backup system for your Git repos 
(e.g. GitHub or GitLab).  Also, for local backups, something solid-state 
like an SSD might be more reliable in the long run.

Thanks,
Neeraj

Re: Git bug report - disk errors on Windows after push

[Ævar Arnfjörð Bjarmason]

On Mon, Apr 18 2022, brian m. carlson wrote:

> [[PGP Signed Part:Undecided]]
> Hey,
>
> On 2022-04-18 at 07:47:07, Attila Csosz wrote:
>> Thank you for filling out a Git bug report!
>> Please answer the following questions to help us understand your issue.
>> 
>> What did you do before the bug happened? (Steps to reproduce your issue)
>> Pushing to a directory directly.
>> 
>> What did you expect to happen? (Expected behavior)
>> Good behavior
>> 
>> What happened instead? (Actual behavior)
>> Disk errors. After pushing to a directory (my origin field is C:\Work for
>> example) the git repository will be corrupted.
>> Moreover causing disk errors not only in the target git bare repository.
>> I've loosed some other files.
>> Cannot be reproduced exactly. However I have decided to report this problem.
>> I've experienced this problem for several years.
>
> There's nothing that Git should be able to do as a normal operating
> system that causes disk errors.  If you're seeing those kinds of errors,
> the likelihood is that your disk is bad and you need to replace it.
>
> Do note that if you're storing your data in a directory managed by a
> cloud syncing service (e.g., Dropbox or OneDrive), then it is known that
> those file systems corrupt Git repositories and you shouldn't use them
> with Git.  That's due to the way they sync data.
>
> However, if this is a regular file system (e.g., NTFS), then Git
> shouldn't be able to cause disk problems, and if it could, that would be
> a serious defect in Windows, so it's almost certainly going to be a
> hardware problem.

I think you're most likely right that this is some local HW issue, but
the report doesn't have enough information to say either way.

But generally as we've been discussing at length in various threads
related to git's fsync() behavior we can and will corrupt repositories
due only to our own (mis)use of FS APIs, i.e. not doing fsyncing and the
like correctly, it's been improved a bit recently.

But we're still in a state of not doing it correctly, and/or relying on
some implicit assumptions.

For something like an external disk that may have its power cable yanked
I'd give it about even odds that it's HW v.s. git's own FS syncing logic
being at fault.

We get away with a lot of shenanigans because we're usually on a
journaled FS, which e.g. on a FAT32 external volume we won't benefit
from.

Attila: I have no idea if this even exists on Windows, but aside from
other suggestions in this thread I'd suggest enabling whatever Window's
equivalent of "sync" mounting is (if it even exists). I.e. to have every
single FS operation by synced to the metal. 

It will make your repository *much much slower*, but it will be safe as
far as any logic errors in git vis-a-vis syncing go.

Re: Git bug report - disk errors on Windows after push

[Attila Csosz]

I'm using git on Windows for 6-7 years sometimes facing this problem
(different computers, different HDDs). Last time the problem occured
on my internal laptop HDD. So I have decided to report the problem.

If unstable to use local bare repos on Windows you should mention in
somewhere or completly disable it. Disk errors are unacceptable even
the git repo is corrupted.

Please check this link also
https://github.community/t/what-could-be-the-reason-my-git-repositories-keep-getting-corrupted-broken/184711

Attila


On Fri, Apr 22, 2022 at 11:27 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Mon, Apr 18 2022, brian m. carlson wrote:
>
> > [[PGP Signed Part:Undecided]]
> > Hey,
> >
> > On 2022-04-18 at 07:47:07, Attila Csosz wrote:
> >> Thank you for filling out a Git bug report!
> >> Please answer the following questions to help us understand your issue.
> >>
> >> What did you do before the bug happened? (Steps to reproduce your issue)
> >> Pushing to a directory directly.
> >>
> >> What did you expect to happen? (Expected behavior)
> >> Good behavior
> >>
> >> What happened instead? (Actual behavior)
> >> Disk errors. After pushing to a directory (my origin field is C:\Work for
> >> example) the git repository will be corrupted.
> >> Moreover causing disk errors not only in the target git bare repository.
> >> I've loosed some other files.
> >> Cannot be reproduced exactly. However I have decided to report this problem.
> >> I've experienced this problem for several years.
> >
> > There's nothing that Git should be able to do as a normal operating
> > system that causes disk errors.  If you're seeing those kinds of errors,
> > the likelihood is that your disk is bad and you need to replace it.
> >
> > Do note that if you're storing your data in a directory managed by a
> > cloud syncing service (e.g., Dropbox or OneDrive), then it is known that
> > those file systems corrupt Git repositories and you shouldn't use them
> > with Git.  That's due to the way they sync data.
> >
> > However, if this is a regular file system (e.g., NTFS), then Git
> > shouldn't be able to cause disk problems, and if it could, that would be
> > a serious defect in Windows, so it's almost certainly going to be a
> > hardware problem.
>
> I think you're most likely right that this is some local HW issue, but
> the report doesn't have enough information to say either way.
>
> But generally as we've been discussing at length in various threads
> related to git's fsync() behavior we can and will corrupt repositories
> due only to our own (mis)use of FS APIs, i.e. not doing fsyncing and the
> like correctly, it's been improved a bit recently.
>
> But we're still in a state of not doing it correctly, and/or relying on
> some implicit assumptions.
>
> For something like an external disk that may have its power cable yanked
> I'd give it about even odds that it's HW v.s. git's own FS syncing logic
> being at fault.
>
> We get away with a lot of shenanigans because we're usually on a
> journaled FS, which e.g. on a FAT32 external volume we won't benefit
> from.
>
> Attila: I have no idea if this even exists on Windows, but aside from
> other suggestions in this thread I'd suggest enabling whatever Window's
> equivalent of "sync" mounting is (if it even exists). I.e. to have every
> single FS operation by synced to the metal.
>
> It will make your repository *much much slower*, but it will be safe as
> far as any logic errors in git vis-a-vis syncing go.

Re: Git bug report - disk errors on Windows after push

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

> For something like an external disk that may have its power cable yanked
> I'd give it about even odds that it's HW v.s. git's own FS syncing logic
> being at fault.

Given that the original report had this:

    Disk errors. After pushing to a directory (my origin field is C:\Work
    for example) the git repository will be corrupted.
    Moreover causing disk errors not only in the target git bare
    repository. I've loosed some other files.

that clearly stated that the corruption spreads outside repositories
managed (and written and fsynced) by Git, it does not seem likely
that Git's relying on "closing is enough as sync daemon will
eventually flush them" is the primary source of corruption.

The recent "are we fsyncing correctly?" topic is about how likely we
would survive in the presense of actions like unplugging without
unmounting.  It does not help a bit against flaky hardware, and even
when it helps, it is merely making the damage caused by such actions
less severe.