recovering from "unordered stage entries in index" error

recovering from "unordered stage entries in index" error

[McHenry, Matt]

	I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

	'git status' shows a few untracked files but is otherwise clean.

	It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).

	Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)

	At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

	(I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)

Re: recovering from "unordered stage entries in index" error

[Junio C Hamano]

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128

Are you by any chance using the split-index mode (which I think is
broken in 2.3.6)?

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

	I had never heard of that.  Google led me to the man page for 'git update-index --split-index', and after reading it, I think this demonstrates that I'm not using that feature:

$ ls .git/*index*
.git/index

> -----Original Message-----
> From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
> Hamano
> Sent: Tuesday, May 19, 2015 11:20
> To: McHenry, Matt
> Cc: git@vger.kernel.org
> Subject: Re: recovering from "unordered stage entries in index" error
> 
> On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
> <mmchenry@carnegielearning.com> wrote:
> >
> >         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now
> unable to run 'git svn fetch' in one of my repositories:
> >
> > $ git svn fetch
> > fatal: unordered stage entries in index
> > write-tree: command returned error: 128
> 
> Are you by any chance using the split-index mode (which I think is
> broken in 2.3.6)?

Re: recovering from "unordered stage entries in index" error

[Duy Nguyen]

On Tue, May 19, 2015 at 8:48 PM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
>
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index

This message can be improved to show what entries have this problem.
But then I don't see any way to recover the index manually. ls-files
will die too. Perhaps we should be gentle in this case: show warnings
instead of aborting the program and internally reorder the index. I
think, unless you have multiple entries with the same stage, the
recovered index should run well. The broken index could be renamed to
index.broken or something for later analysis, or we forbid writing the
reordered index to disk.

Hmm?

> write-tree: command returned error: 128
>
>         'git status' shows a few untracked files but is otherwise clean.
>
>         It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).
>
>         Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)
>
>         At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:
>
> $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
>
>         (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html




-- 
Duy

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

> This message can be improved to show what entries have this problem.

	Yes, that would definitely be a start.  :)

> But then I don't see any way to recover the index manually. ls-files
> will die too. Perhaps we should be gentle in this case: show warnings

	Actually, ls-files succeeds on my broken index:

$ git ls-files > /dev/null
$ echo $?
0

	Could I do something with 'git read-tree' to force creation of a new valid index?  I guess 'git clone' would work too, except that I have 'git svn' metadata that I'd need to preserve.

> instead of aborting the program and internally reorder the index. I
> think, unless you have multiple entries with the same stage, the
> recovered index should run well. The broken index could be renamed to
> index.broken or something for later analysis, or we forbid writing the
> reordered index to disk.
> 
> Hmm?
> 
> > write-tree: command returned error: 128
> >
> >         'git status' shows a few untracked files but is otherwise clean.
> >
> >         It looks like this check was introduced in
> 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
> "read_index_from(): catch out of order entries when reading an index file"
> (first appearing in 2.2.0).
> >
> >         Mailing list discussion looked like it implicated third-party
> tools.  I don't recall running any other tools on this repo; it doesn't do
> much day-to-day other than a long series of 'git svn fetch'es.  (But it's
> been around for a couple of years, so who knows.)
> >
> >         At any rate, what can I do to recover from this situation?  I
> tried to locate a path with multiple index entries like this, but got no
> results:
> >
> > $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
> >
> >         (I originally posted on SO at
> http://stackoverflow.com/questions/30264826/; I'll update that with any
> solutions that come up here, to ease future googling.)
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> 
> --
> Duy

Re: recovering from "unordered stage entries in index" error

[Junio C Hamano]

Duy Nguyen <pclouds@gmail.com> writes:

> This message can be improved to show what entries have this problem.
> But then I don't see any way to recover the index manually. ls-files
> will die too.

Isn't this failure coming from git-svn that tries to write out a
tree after it prepared whatever it wants to record in its (possibly
temporary) index?  I have a feeling that the index held by the end
user is not broken.

Re: recovering from "unordered stage entries in index" error

[Duy Nguyen]

On Thu, May 21, 2015 at 11:49 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Duy Nguyen <pclouds@gmail.com> writes:
>
>> This message can be improved to show what entries have this problem.
>> But then I don't see any way to recover the index manually. ls-files
>> will die too.
>
> Isn't this failure coming from git-svn that tries to write out a
> tree after it prepared whatever it wants to record in its (possibly
> temporary) index?  I have a feeling that the index held by the end
> user is not broken.

Ahh that would explain why ls-files works. Yep.
-- 
Duy

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

> > Isn't this failure coming from git-svn that tries to write out a
> > tree after it prepared whatever it wants to record in its (possibly
> > temporary) index?  I have a feeling that the index held by the end
> > user is not broken.
> 
> Ahh that would explain why ls-files works. Yep.

	I created a copy of this repo + wc via rsync and tried a couple of things.  'git svn rebase -l' worked fine, but didn't fix the error.  Next, reset:

$ git svn log --limit=1 | grep ^r
r231655 | avuong | 2015-05-10 10:32:16 -0400 (Sun, 10 May 2015) | 2 lines

$ git svn reset -r 231655 -p
r231653 = 13a7f6d6a3f3e44ed1c8523b1a63d72fc4f0ddb9 (refs/remotes/trunk)

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

	So it doesn't seem to be specific to the revision being fetched.  I could do a more drastic 'git svn reset', but as you can see I've already fetched a lot of revs, so I'd rather avoid re-fetching if possible.

	Thanks for your help so far -- any other ideas (or requests for further debugging info) are appreciated!

Re: recovering from "unordered stage entries in index" error

[Junio C Hamano]

The message "unordered stage entries in index" comes only when
two adjacent entries in the index are in a wrong order, e.g. "test0"
should come before "test1" but somehow the index records them
in the other way around. Doing something like this:

$ git ls-files >Q
$ LANG=C LC_ALL=C sort Q >R
$ diff Q R

may tell you which entries are wrong, even though it wouldn't show
who made them wrong.

(pardon top-posting, overlong lines and typos; sent from GMail web UI)

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
>
>         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:
>
> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128
>
>         'git status' shows a few untracked files but is otherwise clean.
>
>         It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).
>
>         Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)
>
>         At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:
>
> $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
>
>         (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

	Yes, that does turn up some interesting stuff.  It looks like the repository contains some paths with non-ASCII characters, for example this one has some en-dashes (U+2013) in its name:

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta
Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 – Ninja/FT3 – Ninja__Beta.zip

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta | od -cx
0000000   H   u   r   i   x       w   o   r   k   /   s   o   u   r   c
           7548    6972    2078    6f77    6b72    732f    756f    6372
0000020   e       f   r   o   m       M   a   y       2   0   1   4   /
           2065    7266    6d6f    4d20    7961    3220    3130    2f34
0000040   F   o   r   _   A   n   e   s   h   /   0   6       D   e   l
           6f46    5f72    6e41    7365    2f68    3630    4420    6c65
0000060   i   v   e   r   a   b   l   e   s   /   P   h   a   s   e
           7669    7265    6261    656c    2f73    6850    7361    2065
0000100   2   /   F   T   3     342 200 223       N   i   n   j   a   /
           2f32    5446    2033    80e2    2093    694e    6a6e    2f61
0000120   F   T   3     342 200 223       N   i   n   j   a   _   _   B
           5446    2033    80e2    2093    694e    6a6e    5f61    425f
0000140   e   t   a   .   z   i   p  \n
           7465    2e61    697a    0a70
0000150

	In the output of 'git ls-files', those paths appear quoted (there are almost 100 of them):

$ git ls-files | grep Ninja__Beta
"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

$ git ls-files | grep ^\" | wc -l
89


	In the diff you suggested, 'sort' puts those paths at the absolute top of the list, while plain old ls-files puts them inline with the rest of the contents of the curriculum/ subdir:

$ grep -n Ninja__Beta Q R
Q:36109:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"
R:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

	Also, I have the curriculum/Fluency/ directory marked as sparse-checkout:

$ cat .git/info/sparse-checkout
/*
!/curriculum/Fluency/
!/curriculum/Problems/lisp/
!/curriculum/Problems/lisp_es/
!/curriculum/Problems/sdk/Geometry/
!/curriculum/Problems/sdk_es/Geometry/
!/curriculum/Problems/sdk/Test-Questions/
!/curriculum/Problems/sdk_es/Test-Questions/
!/curriculum/Problems/sdk/Grammar/


	However, I tried to construct a test case that would reproduce this with a simple SVN repo containing a file created by 'touch "make-git-svn-$(echo -e '\u201c')unhappy$(echo -e '\u201d')"', but could not get it to fail.  So there may be something more subtle going on here ...


> -----Original Message-----
> From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
> Hamano
> Sent: Friday, May 22, 2015 15:25
> To: McHenry, Matt
> Cc: git@vger.kernel.org
> Subject: Re: recovering from "unordered stage entries in index" error
> 
> The message "unordered stage entries in index" comes only when
> two adjacent entries in the index are in a wrong order, e.g. "test0"
> should come before "test1" but somehow the index records them
> in the other way around. Doing something like this:
> 
> $ git ls-files >Q
> $ LANG=C LC_ALL=C sort Q >R
> $ diff Q R
> 
> may tell you which entries are wrong, even though it wouldn't show
> who made them wrong.
> 
> (pardon top-posting, overlong lines and typos; sent from GMail web UI)
> 
> On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
> <mmchenry@carnegielearning.com> wrote:
> >
> >         I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now
> unable to run 'git svn fetch' in one of my repositories:
> >
> > $ git svn fetch
> > fatal: unordered stage entries in index
> > write-tree: command returned error: 128
> >
> >         'git status' shows a few untracked files but is otherwise clean.
> >
> >         It looks like this check was introduced in
> 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
> "read_index_from(): catch out of order entries when reading an index file"
> (first appearing in 2.2.0).
> >
> >         Mailing list discussion looked like it implicated third-party
> tools.  I don't recall running any other tools on this repo; it doesn't do
> much day-to-day other than a long series of 'git svn fetch'es.  (But it's
> been around for a couple of years, so who knows.)
> >
> >         At any rate, what can I do to recover from this situation?  I
> tried to locate a path with multiple index entries like this, but got no
> results:
> >
> > $ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '
> >
> >         (I originally posted on SO at
> http://stackoverflow.com/questions/30264826/; I'll update that with any
> solutions that come up here, to ease future googling.)
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: recovering from "unordered stage entries in index" error

[Duy Nguyen]

On Sat, May 23, 2015 at 1:56 AM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
> $ git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128

git-svn does not create the index manually. It uses update-index or
read-tree to do that. While there's still a chance of bugs in
update-index that produces broken index, it's probably read-tree in
this case because it assumes good order from the source tree object,
which is(?) generated by git-svn. And the write-tree message supports
this (the code does read-tree then write-tree).

So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
I'd expect to see something like "git read-tree <sha1>" before "fatal:
unorder...". You can then use git ls-tree <sha1> to examine this tree,
try to sort the file list with "LANG=C sort" and compare with the
original list.
-- 
Duy

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
> I'd expect to see something like "git read-tree <sha1>" before "fatal:
> unorder...". You can then use git ls-tree <sha1> to examine this tree,
> try to sort the file list with "LANG=C sort" and compare with the
> original list.

	There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

$ git svn log -1 --show-commit --oneline trunk
r231655 | 74332b7 | CLT: changed skill from not compound to compound.

	So not surprisingly, I guess, I get basically the same results as with the ls-files commands earlier: files with Unicode chars are quoted and sort at the front:

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree
$ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all
$ grep -n Ninja__Beta ls-tree*
ls-tree:36974:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"
ls-tree-sorted-lc-all:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

	(Just sorting with LANG=C but without LC_ALL=C produced a ton of other differences, mostly around numeric vs. lexical ordering as far as I could tell.)

	I tried this same thing with my test repo, and it exhibits the same quoted-filename-sorts-to-the-top behaviour, but does not exhibit the git svn fetch write-tree error.


$ GIT_TRACE=2 git svn fetch
22:21:16.683918 git.c:557               trace: exec: 'git-svn' 'fetch'
22:21:16.683945 run-command.c:351       trace: run_command: 'git-svn' 'fetch'
02:21:16.918593 git.c:348               trace: built-in: git 'rev-parse' '--git-dir'
02:21:16.920218 git.c:348               trace: built-in: git 'rev-parse' '--show-cdup'
02:21:16.921997 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall'
02:21:16.923609 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.repack'
02:21:16.925164 git.c:348               trace: built-in: git 'config' '--get' 'svn.repackflags'
02:21:16.926706 git.c:348               trace: built-in: git 'config' '--get' 'svn.revision'
02:21:16.928847 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout'
02:21:16.930410 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps'
02:21:16.931963 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.localtime'
02:21:16.933538 git.c:348               trace: built-in: git 'config' '--get' 'svn.includepaths'
02:21:16.935107 git.c:348               trace: built-in: git 'config' '--get' 'svn.username'
02:21:16.936675 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache'
02:21:16.940413 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.quiet'
02:21:16.942064 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor'
02:21:16.943696 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata'
02:21:16.945344 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps'
02:21:16.947607 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.parent'
02:21:16.950737 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom'
02:21:16.952532 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsprog'
02:21:16.954133 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorepaths'
02:21:16.955704 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.followparent'
02:21:16.957287 git.c:348               trace: built-in: git 'config' '--get' 'svn.configdir'
02:21:16.958930 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsfile'
02:21:16.962142 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize'
02:21:16.963913 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorerefs'
02:21:16.966130 git.c:348               trace: built-in: git 'rev-parse' '--symbolic' '--all'
02:21:16.970537 git.c:348               trace: built-in: git 'config' '-l'
02:21:16.972410 git.c:348               trace: built-in: git 'config' '-l'
02:21:16.974187 git.c:348               trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
02:21:16.976074 git.c:348               trace: built-in: git 'config' '-l'
02:21:17.136056 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev'
02:21:17.137928 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev'
02:21:17.140124 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.url'
02:21:17.142192 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl'
02:21:17.144203 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid'
02:21:17.149689 git.c:348               trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--'
02:21:17.152412 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot'
02:21:17.154482 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID'
02:21:17.160391 git.c:348               trace: built-in: git 'cat-file' '--batch'
02:21:17.582641 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655'
02:21:17.585237 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655'
02:21:17.590152 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops'
02:21:17.592063 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata'
02:21:17.593997 git.c:348               trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440'
02:21:17.596529 git.c:348               trace: built-in: git 'write-tree'
fatal: unordered stage entries in index
write-tree: command returned error: 128

Re: recovering from "unordered stage entries in index" error

[Junio C Hamano]

"McHenry, Matt" <mmchenry@carnegielearning.com> writes:

> 	Yes, that does turn up some interesting stuff.  It looks
> 	like the repository contains some paths with non-ASCII
> 	characters, for example this one has some en-dashes (U+2013)
> 	in its name:

Then the recipe in the message you are responding to shoulld be
changed somewhat to tell Git that it is OK to show non-ASCII in
verbatim.

Perhaps doing this once

    git config core.quotepath false

before asking ls-files to show the paths should be sufficient.

Thanks.

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

Per Junio's email, with core.quotepath=false, there are no differences with sorting in either ls-files or the tree named in the GIT_TRACE_2 output:

$ git config --local core.quotepath false

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree
$ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all
$ diff -s ls-tree ls-tree-sorted-lc-all 
Files ls-tree and ls-tree-sorted-lc-all are identical

$ git ls-files > ls-files
$ LANG=C LC_ALL=C sort ls-files > ls-files-sorted-lc-all
$ diff -s ls-files ls-files-sorted-lc-all 
Files ls-files and ls-files-sorted-lc-all are identical


________________________________________
From: McHenry, Matt
Sent: Friday, May 22, 2015 10:47 PM
To: Duy Nguyen
Cc: Junio C Hamano; git@vger.kernel.org
Subject: RE: recovering from "unordered stage entries in index" error

> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
> I'd expect to see something like "git read-tree <sha1>" before "fatal:
> unorder...". You can then use git ls-tree <sha1> to examine this tree,
> try to sort the file list with "LANG=C sort" and compare with the
> original list.

        There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

$ git svn log -1 --show-commit --oneline trunk
r231655 | 74332b7 | CLT: changed skill from not compound to compound.

        So not surprisingly, I guess, I get basically the same results as with the ls-files commands earlier: files with Unicode chars are quoted and sort at the front:

$ git ls-tree --name-only -r 74332b7d653cde7ba3b999cc7b0adcfd9d924440 > ls-tree
$ LANG=C LC_ALL=C sort ls-tree > ls-tree-sorted-lc-all
$ grep -n Ninja__Beta ls-tree*
ls-tree:36974:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"
ls-tree-sorted-lc-all:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

        (Just sorting with LANG=C but without LC_ALL=C produced a ton of other differences, mostly around numeric vs. lexical ordering as far as I could tell.)

        I tried this same thing with my test repo, and it exhibits the same quoted-filename-sorts-to-the-top behaviour, but does not exhibit the git svn fetch write-tree error.


$ GIT_TRACE=2 git svn fetch
22:21:16.683918 git.c:557               trace: exec: 'git-svn' 'fetch'
22:21:16.683945 run-command.c:351       trace: run_command: 'git-svn' 'fetch'
02:21:16.918593 git.c:348               trace: built-in: git 'rev-parse' '--git-dir'
02:21:16.920218 git.c:348               trace: built-in: git 'rev-parse' '--show-cdup'
02:21:16.921997 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall'
02:21:16.923609 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.repack'
02:21:16.925164 git.c:348               trace: built-in: git 'config' '--get' 'svn.repackflags'
02:21:16.926706 git.c:348               trace: built-in: git 'config' '--get' 'svn.revision'
02:21:16.928847 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout'
02:21:16.930410 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps'
02:21:16.931963 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.localtime'
02:21:16.933538 git.c:348               trace: built-in: git 'config' '--get' 'svn.includepaths'
02:21:16.935107 git.c:348               trace: built-in: git 'config' '--get' 'svn.username'
02:21:16.936675 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache'
02:21:16.940413 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.quiet'
02:21:16.942064 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor'
02:21:16.943696 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata'
02:21:16.945344 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps'
02:21:16.947607 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.parent'
02:21:16.950737 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom'
02:21:16.952532 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsprog'
02:21:16.954133 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorepaths'
02:21:16.955704 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.followparent'
02:21:16.957287 git.c:348               trace: built-in: git 'config' '--get' 'svn.configdir'
02:21:16.958930 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsfile'
02:21:16.962142 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize'
02:21:16.963913 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorerefs'
02:21:16.966130 git.c:348               trace: built-in: git 'rev-parse' '--symbolic' '--all'
02:21:16.970537 git.c:348               trace: built-in: git 'config' '-l'
02:21:16.972410 git.c:348               trace: built-in: git 'config' '-l'
02:21:16.974187 git.c:348               trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
02:21:16.976074 git.c:348               trace: built-in: git 'config' '-l'
02:21:17.136056 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev'
02:21:17.137928 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev'
02:21:17.140124 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.url'
02:21:17.142192 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl'
02:21:17.144203 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid'
02:21:17.149689 git.c:348               trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--'
02:21:17.152412 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot'
02:21:17.154482 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID'
02:21:17.160391 git.c:348               trace: built-in: git 'cat-file' '--batch'
02:21:17.582641 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655'
02:21:17.585237 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655'
02:21:17.590152 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops'
02:21:17.592063 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata'
02:21:17.593997 git.c:348               trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440'
02:21:17.596529 git.c:348               trace: built-in: git 'write-tree'
fatal: unordered stage entries in index
write-tree: command returned error: 128

Re: recovering from "unordered stage entries in index" error

[Duy Nguyen]

On Sat, May 23, 2015 at 9:47 AM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
>> So maybe you can do "GIT_TRACE=2 git svn fetch" and post the output.
>> I'd expect to see something like "git read-tree <sha1>" before "fatal:
>> unorder...". You can then use git ls-tree <sha1> to examine this tree,
>> try to sort the file list with "LANG=C sort" and compare with the
>> original list.
>
>         There is no read-tree in the output (below).  The sha1 that is mentioned, 74332b7, is the one for the current trunk:

Hm.. neither read-tree nor update-index is in the output. I can see
git-svn closing stderr sometimes, but not sure if that's why we don't
see these commands, or something else. Could you try again with
GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post
the content of /abso../some/where?
-- 
Duy

RE: recovering from "unordered stage entries in index" error

[McHenry, Matt]

> see these commands, or something else. Could you try again with
> GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post
> the content of /abso../some/where?

	It looks the same as far as I can see:

$ GIT_TRACE=/tmp/git-trace git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

$ egrep -i 'read|tree|update|index' /tmp/git-trace
13:26:08.169921 git.c:348               trace: built-in: git 'write-tree'

$ cat /tmp/git-trace
09:26:07.402735 git.c:557               trace: exec: 'git-svn' 'fetch'
09:26:07.402784 run-command.c:351       trace: run_command: 'git-svn' 'fetch'
09:26:07.688866 git.c:348               trace: built-in: git 'rev-parse' '--show-prefix'
13:26:07.691207 git.c:348               trace: built-in: git 'rev-parse' '--git-dir'
13:26:07.693228 git.c:348               trace: built-in: git 'rev-parse' '--show-cdup'
13:26:07.695594 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.quiet'
13:26:07.697279 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsprog'
13:26:07.699030 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorepaths'
13:26:07.700914 git.c:348               trace: built-in: git 'config' '--get' 'svn.authorsfile'
13:26:07.702872 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.followparent'
13:26:07.704949 git.c:348               trace: built-in: git 'config' '--get' 'svn.configdir'
13:26:07.706674 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.localtime'
13:26:07.708590 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvmProps'
13:26:07.710669 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.useSvnsyncProps'
13:26:07.712602 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noauthcache'
13:26:07.714527 git.c:348               trace: built-in: git 'config' '--get' 'svn.username'
13:26:07.716257 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.logwindowsize'
13:26:07.717872 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.noMetadata'
13:26:07.719905 git.c:348               trace: built-in: git 'config' '--get' 'svn.ignorerefs'
13:26:07.721623 git.c:348               trace: built-in: git 'config' '--get' 'svn.includepaths'
13:26:07.723485 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.nocheckout'
13:26:07.725441 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.addauthorfrom'
13:26:07.727125 git.c:348               trace: built-in: git 'config' '--get' 'svn.repackflags'
13:26:07.729179 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn.repack'
13:26:07.731143 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.uselogauthor'
13:26:07.733139 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.parent'
13:26:07.734911 git.c:348               trace: built-in: git 'config' '--bool' '--get' 'svn.fetchall'
13:26:07.736992 git.c:348               trace: built-in: git 'config' '--get' 'svn.revision'
13:26:07.739398 git.c:348               trace: built-in: git 'rev-parse' '--symbolic' '--all'
13:26:07.744514 git.c:348               trace: built-in: git 'config' '-l'
13:26:07.746770 git.c:348               trace: built-in: git 'config' '-l'
13:26:07.749108 git.c:348               trace: built-in: git 'config' '--bool' 'svn.useSvmProps'
13:26:07.751613 git.c:348               trace: built-in: git 'config' '-l'
13:26:07.907707 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.branches-maxRev'
13:26:07.910171 git.c:348               trace: built-in: git 'config' '--int' '--get' 'svn-remote.svn.tags-maxRev'
13:26:07.912608 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.url'
13:26:07.915539 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.pushurl'
13:26:07.917825 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.uuid'
13:26:07.919620 git.c:348               trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/trunk^0'
13:26:07.923752 git.c:348               trace: built-in: git 'rev-list' '--pretty=raw' '--reverse' '74332b7d653cde7ba3b999cc7b0adcfd9d924440..refs/remotes/trunk' '--'
13:26:07.926128 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteRoot'
13:26:07.928707 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.rewriteUUID'
13:26:07.933621 git.c:348               trace: built-in: git 'cat-file' '--batch'
13:26:08.150396 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.branches-maxRev' '231655'
13:26:08.152963 git.c:348               trace: built-in: git 'config' 'svn-remote.svn.tags-maxRev' '231655'
13:26:08.155457 git.c:348               trace: built-in: git 'rev-parse' '--verify' 'refs/remotes/trunk^0'
13:26:08.158789 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.usesvmprops'
13:26:08.160929 git.c:348               trace: built-in: git 'config' '--get' 'svn-remote.svn.nometadata'
13:26:08.164778 git.c:348               trace: built-in: git 'cat-file' 'commit' '74332b7d653cde7ba3b999cc7b0adcfd9d924440'
13:26:08.169921 git.c:348               trace: built-in: git 'write-tree'

Re: recovering from "unordered stage entries in index" error

[Duy Nguyen]

On Tue, May 26, 2015 at 8:28 PM, McHenry, Matt
<mmchenry@carnegielearning.com> wrote:
>> see these commands, or something else. Could you try again with
>> GIT_TRACE=/absolute/path/to/some/where instead of GIT_TRACE=2 and post
>> the content of /abso../some/where?
>
>         It looks the same as far as I can see:
>
> $ GIT_TRACE=/tmp/git-trace git svn fetch
> fatal: unordered stage entries in index
> write-tree: command returned error: 128
>
> $ egrep -i 'read|tree|update|index' /tmp/git-trace
> 13:26:08.169921 git.c:348               trace: built-in: git 'write-tree'

OK I give up. Can't think of how the index is written, and by whom.
-- 
Duy