HFS+ Unicode weirdness

HFS+ Unicode weirdness

[Wincent Colaiuta]

On a brand new clone of git.git the file "gitweb/test/Märchen" is  
provoking some weird behaviour running on Mac OS X and the toy HFS+  
filesystem. Note how the unmodified checkout of the file is shown as  
"untracked" by "git st", but on deleting the file it's shown as  
"deleted". If I build a copy of Git based on the clean working tree  
then the resulting build has a version number of "1.5.x-dirty".

Any suggestions on where to start investigating the cause of this?  
About the only lead I have is that if I create a file with that name  
by typing it's name  it's encoded as "Ma\314\210rchen", but the file  
in the git.git repo is encoded as "M\303\244rchen".

Cheers,
Wincent

$ git clone git://git.kernel.org/pub/scm/git/git.git
Initialized empty Git repository in /tmp/git/.git/
remote: Generating pack...
remote: Counting objects: 12259
Done counting 58277 objects.
remote: Deltifying 58277 objects...
remote:  100% (58277/58277) done
Indexing 58277 objects...
remote: Total 58277 (delta 40699), reused 58034 (delta 40521)
  100% (58277/58277) done
Resolving 40699 deltas...
  100% (40699/40699) done
$ cd git
/tmp/git
$ git st
# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#       gitweb/test/Märchen
nothing added to commit but untracked files present (use "git add" to  
track)
$ ls -laF gitweb/test
total 24
drwxr-xr-x   5 wincent  wheel  170 Sep  4 14:18 ./
drwxr-xr-x   9 wincent  wheel  306 Sep  4 14:18 ../
-rw-r--r--   1 wincent  wheel   17 Sep  4 14:18 Ma??rchen
-rw-r--r--   1 wincent  wheel   31 Sep  4 14:18 file with spaces
-rw-r--r--   1 wincent  wheel   37 Sep  4 14:18 file+plus+sign
$ rm gitweb/test/Märchen
$ git st
# On branch master
# Changed but not updated:
#   (use "git add/rm <file>..." to update what will be committed)
#
#       deleted:    gitweb/test/Märchen
#
no changes added to commit (use "git add" and/or "git commit -a")

Re: HFS+ Unicode weirdness

[Johannes Schindelin]

Hi,

On Tue, 4 Sep 2007, Wincent Colaiuta wrote:

> On a brand new clone of git.git the file "gitweb/test/M?rchen" is provoking
> some weird behaviour running on Mac OS X and the toy HFS+ filesystem.

Please search the mail archives.  This has come up quite a couple of 
times.

Ciao,
Dscho

Re: HFS+ Unicode weirdness

[Wincent Colaiuta]

El 4/9/2007, a las 15:00, Johannes Schindelin escribió:

> Hi,
>
> On Tue, 4 Sep 2007, Wincent Colaiuta wrote:
>
>> On a brand new clone of git.git the file "gitweb/test/M?rchen" is  
>> provoking
>> some weird behaviour running on Mac OS X and the toy HFS+ filesystem.
>
> Please search the mail archives.  This has come up quite a couple of
> times.

Finally found it: for the record this thread seems to be relevant:

<http://marc.info/?l=git&m=117053359117549&w=2>

For what it's worth Subversion has exactly the same kind of problem  
when only some developers in a team work on HFS. For example:

<http://svn.haxx.se/users/archive-2005-12/0373.shtml>
<http://subversion.tigris.org/issues/show_bug.cgi?id=2464>

Hopefully the wait for Apple to release a non-broken filesystem won't  
be too long, ZFS being the most likely candidate at this stage.

Cheers,
Wincent

Re: HFS+ Unicode weirdness

[David Symonds]

On 04/09/07, Wincent Colaiuta <win@wincent.com> wrote:
> On a brand new clone of git.git the file "gitweb/test/Märchen" is
> provoking some weird behaviour running on Mac OS X and the toy HFS+
> filesystem. Note how the unmodified checkout of the file is shown as
> "untracked" by "git st", but on deleting the file it's shown as
> "deleted". If I build a copy of Git based on the clean working tree
> then the resulting build has a version number of "1.5.x-dirty".
>
> Any suggestions on where to start investigating the cause of this?
> About the only lead I have is that if I create a file with that name
> by typing it's name  it's encoded as "Ma\314\210rchen", but the file
> in the git.git repo is encoded as "M\303\244rchen".

It's because OS X is using composite Unicode encoding (storing "ä" as
"a" + umlaut). Git doesn't canonicalise things, since OS X  does its
jiggery-pokery of that kind of stuff in CoreFramework, which I believe
Git doesn't use.


Dave.