* New repo quickly corrupted
@ 2007-11-15 21:50 Jason Sewall
2007-11-15 21:55 ` Junio C Hamano
2007-11-15 22:01 ` Linus Torvalds
0 siblings, 2 replies; 11+ messages in thread
From: Jason Sewall @ 2007-11-15 21:50 UTC (permalink / raw
To: git, Junio C Hamano
I was working on a new repo created from a tar snapshot of an old
project, and after about a half-hour and about 5 commits for hacking,
I had fixed the problem I set out to take care of.
However, I tried to run git gc to clean it up and put it away and I
was met with the following:
[sewall@hobo SWIFT++_1.2 (master)]$ git gc
Counting objects: 227, done.
error: corrupt loose object '680aba836639d14a36b81f9e29da52bd9af69770'
fatal: object 680aba836639d14a36b81f9e29da52bd9af69770 cannot be read
error: failed to run repack
I've followed discussions on the list about corrupted repos and did
[sewall@hobo SWIFT++_1.2 (master)]$ git fsck
error: corrupt loose object '24bbb41919ef906db6f40c3c80f246b7e0cdf9f4'
error: 24bbb41919ef906db6f40c3c80f246b7e0cdf9f4: object corrupt or missing
error: corrupt loose object '680aba836639d14a36b81f9e29da52bd9af69770'
error: 680aba836639d14a36b81f9e29da52bd9af69770: object corrupt or missing
error: 8de8382bdc3cc5f936670df414b33ee63927f3a4: object corrupt or missing
error: corrupt loose object 'd186d23e87ddf344acc56f48e3cf2f61c7a47e16'
error: d186d23e87ddf344acc56f48e3cf2f61c7a47e16: object corrupt or missing
missing blob 680aba836639d14a36b81f9e29da52bd9af69770
missing blob 8de8382bdc3cc5f936670df414b33ee63927f3a4
missing blob d186d23e87ddf344acc56f48e3cf2f61c7a47e16
missing blob 24bbb41919ef906db6f40c3c80f246b7e0cdf9f4
And since these were all blobs, I looked for em with git-log:
[sewall@hobo SWIFT++_1.2 (master)]$ git log --raw --all | grep 24bbb*
[sewall@hobo SWIFT++_1.2 (master)]$ git log --raw --all | grep d186*
[sewall@hobo SWIFT++_1.2 (master)]$ git log --raw --all | grep 8de83*
[sewall@hobo SWIFT++_1.2 (master)]$ git log --raw --all | grep 680ab*
So it would assume that these are dangling objects. Given the simple
history of the repo, the only think I can think of that could have
caused it would be that my first commit was of the original files, and
I immediately remembered that I wanted to run dos2unix on the files so
I did and did a commit -a --amend.
I untarred the original snapshot again and ran git-hash-object over
all the files, and sure enough each of those missing hashes
corresponds to a blob in the original (non-dos2unix'd) file.
I copied the corrupted repo, manually deleted those objects, then
hash-object -w the appropriate originals back in. git-fsck was
mollified.
Finally, I tried repeating the process and the same error occurs.
In summary:
unzip original.zip
git init
git add .
git commit -m "Initial commit"
find . -type f -exec dos2unix {} +
git commit -a --amend
[accept same message]
git fsck
[same error message as above]
I just built my git today, unmodified:
git version 1.5.3.5.721.g039b
What's going on?
Jason
P.S. These files aren't particularly secret, so I'm happy to post them
if need be. Its actually someone else's code (a fellow student who
graduated some time before I arrived in grad school); I was asked to
modernize it enough to compile on on the latest gcc.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 21:50 New repo quickly corrupted Jason Sewall
@ 2007-11-15 21:55 ` Junio C Hamano
2007-11-15 21:59 ` Jason Sewall
2007-11-15 22:01 ` Linus Torvalds
1 sibling, 1 reply; 11+ messages in thread
From: Junio C Hamano @ 2007-11-15 21:55 UTC (permalink / raw
To: Jason Sewall; +Cc: git, Junio C Hamano
"Jason Sewall" <jasonsewall@gmail.com> writes:
> find . -type f -exec dos2unix {} +
> git commit -a --amend
> [accept same message]
> git fsck
> [same error message as above]
>
> I just built my git today, unmodified:
> git version 1.5.3.5.721.g039b
>
> What's going on?
Carelessness is what is going on.
Notice how nicely your "find . -type f -exec dos2unix {}" goes
down to .git/objects and eats your loose objects
You probably wanted to do
git ls-files | xargs dos2unix
instead.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 21:55 ` Junio C Hamano
@ 2007-11-15 21:59 ` Jason Sewall
0 siblings, 0 replies; 11+ messages in thread
From: Jason Sewall @ 2007-11-15 21:59 UTC (permalink / raw
To: Junio C Hamano; +Cc: git
On Nov 15, 2007 4:55 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Jason Sewall" <jasonsewall@gmail.com> writes:
>
> > find . -type f -exec dos2unix {} +
> > git commit -a --amend
> > [accept same message]
> > git fsck
> > [same error message as above]
> >
> > I just built my git today, unmodified:
> > git version 1.5.3.5.721.g039b
> >
> > What's going on?
>
> Carelessness is what is going on.
>
> Notice how nicely your "find . -type f -exec dos2unix {}" goes
> down to .git/objects and eats your loose objects
>
> You probably wanted to do
>
> git ls-files | xargs dos2unix
>
> instead.
>
Sure enough. I'll chalk that one up to too many mind-numbing meetings today.
Thanks for the wake-up...
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 21:50 New repo quickly corrupted Jason Sewall
2007-11-15 21:55 ` Junio C Hamano
@ 2007-11-15 22:01 ` Linus Torvalds
2007-11-15 22:06 ` Jason Sewall
2007-11-15 22:12 ` Nicolas Pitre
1 sibling, 2 replies; 11+ messages in thread
From: Linus Torvalds @ 2007-11-15 22:01 UTC (permalink / raw
To: Jason Sewall; +Cc: git, Junio C Hamano
On Thu, 15 Nov 2007, Jason Sewall wrote:
>
> find . -type f -exec dos2unix {} +
Oops.
You just ran "dos2unix" on all the *git* files!
Which certainly explains why all your old objects got corrupted!
> What's going on?
PEBKAC ;)
To avoid this error in the future, may I suggest using
git ls-files -z | xargs -0 dos2unix
or similar. Please DO NOT write to the .git/objects directory.
Linus
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 22:01 ` Linus Torvalds
@ 2007-11-15 22:06 ` Jason Sewall
2007-11-15 22:12 ` Nicolas Pitre
1 sibling, 0 replies; 11+ messages in thread
From: Jason Sewall @ 2007-11-15 22:06 UTC (permalink / raw
To: Linus Torvalds; +Cc: git, Junio C Hamano
On Nov 15, 2007 5:01 PM, Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
>
> On Thu, 15 Nov 2007, Jason Sewall wrote:
> >
> > find . -type f -exec dos2unix {} +
>
> Oops.
>
> You just ran "dos2unix" on all the *git* files!
>
> Which certainly explains why all your old objects got corrupted!
>
> > What's going on?
>
> PEBKAC ;)
Yeah, Junio beat you to it. :)
I take comfort in the fact that others have suffered far worse
consequences from find misuse....
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 22:01 ` Linus Torvalds
2007-11-15 22:06 ` Jason Sewall
@ 2007-11-15 22:12 ` Nicolas Pitre
2007-11-15 22:37 ` Linus Torvalds
1 sibling, 1 reply; 11+ messages in thread
From: Nicolas Pitre @ 2007-11-15 22:12 UTC (permalink / raw
To: Linus Torvalds; +Cc: Jason Sewall, git, Junio C Hamano
On Thu, 15 Nov 2007, Linus Torvalds wrote:
>
>
> On Thu, 15 Nov 2007, Jason Sewall wrote:
> >
> > find . -type f -exec dos2unix {} +
>
> Oops.
>
> You just ran "dos2unix" on all the *git* files!
Does "dos2unix" override file access bits? Because the object store is
always made read-only.
Nicolas
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 22:12 ` Nicolas Pitre
@ 2007-11-15 22:37 ` Linus Torvalds
2007-11-16 5:45 ` Christian Couder
0 siblings, 1 reply; 11+ messages in thread
From: Linus Torvalds @ 2007-11-15 22:37 UTC (permalink / raw
To: Nicolas Pitre; +Cc: Jason Sewall, git, Junio C Hamano
On Thu, 15 Nov 2007, Nicolas Pitre wrote:
>
> Does "dos2unix" override file access bits? Because the object store is
> always made read-only.
Almost all programs like that will entirely ignor the fact that something
is read-only.
Why? Becuase you end up having to create a new file *anyway*. So nobody
does modify-in-place, they literally end up doing
- create temp-file
- while (data)
read old file, write to tempfile
- rename temp-file over oldfile
and unless you *explicitly* look at the permission bits you'll never even
notice that the old file was read-only, because none of the steps above
care at all!
So marking things read-only will give only limited protection. It will
protect against most editors, and will protect against things that change
files in-place and literally try to open the original file as read-write,
but not much else.
Linus
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-15 22:37 ` Linus Torvalds
@ 2007-11-16 5:45 ` Christian Couder
2007-11-16 7:35 ` Andreas Ericsson
0 siblings, 1 reply; 11+ messages in thread
From: Christian Couder @ 2007-11-16 5:45 UTC (permalink / raw
To: Linus Torvalds; +Cc: Nicolas Pitre, Jason Sewall, git, Junio C Hamano
Le jeudi 15 novembre 2007, Linus Torvalds a écrit :
> On Thu, 15 Nov 2007, Nicolas Pitre wrote:
> > Does "dos2unix" override file access bits? Because the object store is
> > always made read-only.
>
> Almost all programs like that will entirely ignor the fact that something
> is read-only.
What if the .git/objects/ sudirectories were also read-only ?
Christian.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-16 5:45 ` Christian Couder
@ 2007-11-16 7:35 ` Andreas Ericsson
2007-11-17 12:53 ` Christian Couder
0 siblings, 1 reply; 11+ messages in thread
From: Andreas Ericsson @ 2007-11-16 7:35 UTC (permalink / raw
To: Christian Couder
Cc: Linus Torvalds, Nicolas Pitre, Jason Sewall, git, Junio C Hamano
Christian Couder wrote:
> Le jeudi 15 novembre 2007, Linus Torvalds a écrit :
>> On Thu, 15 Nov 2007, Nicolas Pitre wrote:
>>> Does "dos2unix" override file access bits? Because the object store is
>>> always made read-only.
>> Almost all programs like that will entirely ignor the fact that something
>> is read-only.
>
> What if the .git/objects/ sudirectories were also read-only ?
>
Then git wouldn't be able to write to it without chmod()'ing it each time.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-16 7:35 ` Andreas Ericsson
@ 2007-11-17 12:53 ` Christian Couder
2007-11-17 14:13 ` Robin Rosenberg
0 siblings, 1 reply; 11+ messages in thread
From: Christian Couder @ 2007-11-17 12:53 UTC (permalink / raw
To: Andreas Ericsson
Cc: Linus Torvalds, Nicolas Pitre, Jason Sewall, git, Junio C Hamano
Le vendredi 16 novembre 2007, Andreas Ericsson a écrit :
> Christian Couder wrote:
> > Le jeudi 15 novembre 2007, Linus Torvalds a écrit :
> >> On Thu, 15 Nov 2007, Nicolas Pitre wrote:
> >>> Does "dos2unix" override file access bits? Because the object store
> >>> is always made read-only.
> >>
> >> Almost all programs like that will entirely ignor the fact that
> >> something is read-only.
> >
> > What if the .git/objects/ sudirectories were also read-only ?
>
> Then git wouldn't be able to write to it without chmod()'ing it each
> time.
Yes, but some (not manly enough) people might want the extra safety even if
it means a performance penalty.
Christian.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: New repo quickly corrupted
2007-11-17 12:53 ` Christian Couder
@ 2007-11-17 14:13 ` Robin Rosenberg
0 siblings, 0 replies; 11+ messages in thread
From: Robin Rosenberg @ 2007-11-17 14:13 UTC (permalink / raw
To: Christian Couder
Cc: Andreas Ericsson, Linus Torvalds, Nicolas Pitre, Jason Sewall,
git, Junio C Hamano
lördag 17 november 2007 skrev Christian Couder:
> Le vendredi 16 novembre 2007, Andreas Ericsson a écrit :
> > Christian Couder wrote:
> > > Le jeudi 15 novembre 2007, Linus Torvalds a écrit :
> > >> On Thu, 15 Nov 2007, Nicolas Pitre wrote:
> > >>> Does "dos2unix" override file access bits? Because the object store
> > >>> is always made read-only.
> > >>
> > >> Almost all programs like that will entirely ignor the fact that
> > >> something is read-only.
> > >
> > > What if the .git/objects/ sudirectories were also read-only ?
> >
> > Then git wouldn't be able to write to it without chmod()'ing it each
> > time.
>
> Yes, but some (not manly enough) people might want the extra safety even if
> it means a performance penalty.
Those do manly enough to do run find . -exec ;)
-- robin
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2007-11-17 14:11 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-11-15 21:50 New repo quickly corrupted Jason Sewall
2007-11-15 21:55 ` Junio C Hamano
2007-11-15 21:59 ` Jason Sewall
2007-11-15 22:01 ` Linus Torvalds
2007-11-15 22:06 ` Jason Sewall
2007-11-15 22:12 ` Nicolas Pitre
2007-11-15 22:37 ` Linus Torvalds
2007-11-16 5:45 ` Christian Couder
2007-11-16 7:35 ` Andreas Ericsson
2007-11-17 12:53 ` Christian Couder
2007-11-17 14:13 ` Robin Rosenberg
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).