Git 2.26.2 and failed self tests on OS X

Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

Hi Everyone,

I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
failing.

The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.

Jeff

*** t3902-quoted.sh ***
not ok 1 - setup
#
#
#               mkdir "$FN" &&
#               for_each_name "echo initial >\"\$name\"" &&
#               git add . &&
#               git commit -q -m Initial &&
#
#               for_each_name "echo second >\"\$name\"" &&
#               git commit -a -m Second &&
#
#               for_each_name "echo modified >\"\$name\""
#
#
ok 2 - setup expected files
not ok 3 - check fully quoted output from ls-files
#
#
#               git ls-files >current && test_cmp expect.quoted current
#
#
not ok 4 - check fully quoted output from diff-files
#
#
#               git diff --name-only >current &&
#               test_cmp expect.quoted current
#
#
not ok 5 - check fully quoted output from diff-index
#
#
#               git diff --name-only HEAD >current &&
#               test_cmp expect.quoted current
#
#
not ok 6 - check fully quoted output from diff-tree
#
#
#               git diff --name-only HEAD^ HEAD >current &&
#               test_cmp expect.quoted current
#
#
not ok 7 - check fully quoted output from ls-tree
#
#
#               git ls-tree --name-only -r HEAD >current &&
#               test_cmp expect.quoted current
#
#
ok 8 - setting core.quotepath
not ok 9 - check fully quoted output from ls-files
#
#
#               git ls-files >current && test_cmp expect.raw current
#
#
not ok 10 - check fully quoted output from diff-files
#
#
#               git diff --name-only >current &&
#               test_cmp expect.raw current
#
#
not ok 11 - check fully quoted output from diff-index
#
#
#               git diff --name-only HEAD >current &&
#               test_cmp expect.raw current
#
#
not ok 12 - check fully quoted output from diff-tree
#
#
#               git diff --name-only HEAD^ HEAD >current &&
#               test_cmp expect.raw current
#
#
not ok 13 - check fully quoted output from ls-tree
#
#
#               git ls-tree --name-only -r HEAD >current &&
#               test_cmp expect.raw current
#
#
# failed 11 among 13 test(s)
1..13
make[2]: *** [t3902-quoted.sh] Error 1
make[1]: *** [test] Error 2
make: *** [test] Error 2

Re: Git 2.26.2 and failed self tests on OS X

[Eric Sunshine]

> I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> failing.
>
> *** t3902-quoted.sh ***
> not ok 1 - setup
> not ok 3 - check fully quoted output from ls-files
> not ok 4 - check fully quoted output from diff-files
> not ok 5 - check fully quoted output from diff-index
> not ok 6 - check fully quoted output from diff-tree
> not ok 7 - check fully quoted output from ls-tree
> not ok 9 - check fully quoted output from ls-files
> not ok 10 - check fully quoted output from diff-files
> not ok 11 - check fully quoted output from diff-index
> not ok 12 - check fully quoted output from diff-tree
> not ok 13 - check fully quoted output from ls-tree

I am unable to reproduce these failures on OS X 10.13.6.

Re: Git 2.26.2 and failed self tests on OS X

[Carlo Marcelo Arenas Belón]

On Mon, May 04, 2020 at 03:03:01AM -0400, Jeffrey Walton wrote:
> 
> I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> failing.
> 
> The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.

if the problem is the bash version but you can build/install a newer one that
doesn't have this problem then you can get the test to use it by pointing to
it with TEST_SHELL_PATH as in :

  $ make TEST_SHELL_PATH=/usr/local/bin/bash test

Carlo

Re: Git 2.26.2 and failed self tests on OS X

[Eric Sunshine]

On Mon, May 4, 2020 at 12:52 PM Carlo Marcelo Arenas Belón
<carenas@gmail.com> wrote:
> On Mon, May 04, 2020 at 03:03:01AM -0400, Jeffrey Walton wrote:
> > I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> > and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> > failing.
> >
> > The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.
>
> if the problem is the bash version but you can build/install a newer one that
> doesn't have this problem [...]

My OS X 10.13.6 has Bash 3.2.57, as well, and I'm unable to reproduce
these failures, so it seems fairly unlikely that the problem Jeffrey
is seeing isn't tied to Bash.

Re: Git 2.26.2 and failed self tests on OS X

[Torsten Bögershausen]

On Mon, May 04, 2020 at 01:18:06PM -0400, Eric Sunshine wrote:
> On Mon, May 4, 2020 at 12:52 PM Carlo Marcelo Arenas Belón
> <carenas@gmail.com> wrote:
> > On Mon, May 04, 2020 at 03:03:01AM -0400, Jeffrey Walton wrote:
> > > I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> > > and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> > > failing.
> > >
> > > The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.
> >
> > if the problem is the bash version but you can build/install a newer one that
> > doesn't have this problem [...]
>
> My OS X 10.13.6 has Bash 3.2.57, as well, and I'm unable to reproduce
> these failures, so it seems fairly unlikely that the problem Jeffrey
> is seeing isn't tied to Bash.

Should I read this as
".. seems fairly likely that the problem Jeffrey is seeing is tied to bash" ?

I can't reproduce it either, so my reasoning would rather be
"The problem is probably not caused by the shipped bash version"

It could be something in the environemt, that confuses our tests here.

Does
./t3902-quoted.sh   -v -d
give anything useful ?

Re: Git 2.26.2 and failed self tests on OS X

[Eric Sunshine]

On Mon, May 4, 2020 at 3:08 PM Torsten Bögershausen <tboegi@web.de> wrote:
> On Mon, May 04, 2020 at 01:18:06PM -0400, Eric Sunshine wrote:
> > My OS X 10.13.6 has Bash 3.2.57, as well, and I'm unable to reproduce
> > these failures, so it seems fairly unlikely that the problem Jeffrey
> > is seeing isn't tied to Bash.
>
> Should I read this as
> ".. seems fairly likely that the problem Jeffrey is seeing is tied to bash" ?

Yes, applying s/unlikely/likely/ to my statement is the correct
interpretation. (Last-second editing strikes again.)

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Mon, May 4, 2020 at 3:08 PM Torsten Bögershausen <tboegi@web.de> wrote:
>
> On Mon, May 04, 2020 at 01:18:06PM -0400, Eric Sunshine wrote:
> > On Mon, May 4, 2020 at 12:52 PM Carlo Marcelo Arenas Belón
> > <carenas@gmail.com> wrote:
> > > On Mon, May 04, 2020 at 03:03:01AM -0400, Jeffrey Walton wrote:
> > > > I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> > > > and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> > > > failing.
> > > >
> > > > The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.
> > >
> > > if the problem is the bash version but you can build/install a newer one that
> > > doesn't have this problem [...]
> >
> > My OS X 10.13.6 has Bash 3.2.57, as well, and I'm unable to reproduce
> > these failures, so it seems fairly unlikely that the problem Jeffrey
> > is seeing isn't tied to Bash.
>
> Should I read this as
> ".. seems fairly likely that the problem Jeffrey is seeing is tied to bash" ?
>
> I can't reproduce it either, so my reasoning would rather be
> "The problem is probably not caused by the shipped bash version"
>
> It could be something in the environemt, that confuses our tests here.
>
> Does
> ./t3902-quoted.sh   -v -d
> give anything useful ?

Yes, it looks like the command reveals a couple of problems.

I'll have to look into my iconv recipe. It has not changed in a while,
so something else may be responsible for the failure.
https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.

I'm not sure what to do with the HEAD problems.

$ ./t3902-quoted.sh   -v -d
Initialized empty Git repository in
/Users/jwalton/Build-Scripts/git-2.26.2/t/trash
directory.t3902-quoted/.git/
expecting success of 3902.1 'setup':

        mkdir "$FN" &&
        for_each_name "echo initial >\"\$name\"" &&
        git add . &&
        git commit -q -m Initial &&

        for_each_name "echo second >\"\$name\"" &&
        git commit -a -m Second &&

        for_each_name "echo modified >\"\$name\""


fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
    precomposed unicode is not supported.
    If you want to use decomposed unicode, run
    "git config core.precomposeunicode false"

not ok 1 - setup
#
#
#               mkdir "$FN" &&
#               for_each_name "echo initial >\"\$name\"" &&
#               git add . &&
#               git commit -q -m Initial &&
#
#               for_each_name "echo second >\"\$name\"" &&
#               git commit -a -m Second &&
#
#               for_each_name "echo modified >\"\$name\""
#
#

expecting success of 3902.2 'setup expected files':
cat >expect.quoted <<\EOF &&
Name
"Name and a\nLF"
"Name and an\tHT"
"Name\""
With SP in it
"\346\277\261\351\207\216\t\347\264\224"
"\346\277\261\351\207\216\n\347\264\224"
"\346\277\261\351\207\216 \347\264\224"
"\346\277\261\351\207\216\"\347\264\224"
"\346\277\261\351\207\216/file"
"\346\277\261\351\207\216\347\264\224"
EOF

cat >expect.raw <<\EOF
Name
"Name and a\nLF"
"Name and an\tHT"
"Name\""
With SP in it
"濱野\t純"
"濱野\n純"
濱野 純
"濱野\"純"
濱野/file
濱野純
EOF

ok 2 - setup expected files

expecting success of 3902.3 'check fully quoted output from ls-files':

        git ls-files >current && test_cmp expect.quoted current


--- expect.quoted       2020-05-04 19:14:39.000000000 +0000
+++ current     2020-05-04 19:14:39.000000000 +0000
@@ -1,11 +0,0 @@
-Name
-"Name and a\nLF"
-"Name and an\tHT"
-"Name\""
-With SP in it
-"\346\277\261\351\207\216\t\347\264\224"
-"\346\277\261\351\207\216\n\347\264\224"
-"\346\277\261\351\207\216 \347\264\224"
-"\346\277\261\351\207\216\"\347\264\224"
-"\346\277\261\351\207\216/file"
-"\346\277\261\351\207\216\347\264\224"
not ok 3 - check fully quoted output from ls-files
#
#
#               git ls-files >current && test_cmp expect.quoted current
#
#

expecting success of 3902.4 'check fully quoted output from diff-files':

        git diff --name-only >current &&
        test_cmp expect.quoted current


--- expect.quoted       2020-05-04 19:14:39.000000000 +0000
+++ current     2020-05-04 19:14:39.000000000 +0000
@@ -1,11 +0,0 @@
-Name
-"Name and a\nLF"
-"Name and an\tHT"
-"Name\""
-With SP in it
-"\346\277\261\351\207\216\t\347\264\224"
-"\346\277\261\351\207\216\n\347\264\224"
-"\346\277\261\351\207\216 \347\264\224"
-"\346\277\261\351\207\216\"\347\264\224"
-"\346\277\261\351\207\216/file"
-"\346\277\261\351\207\216\347\264\224"
not ok 4 - check fully quoted output from diff-files
#
#
#               git diff --name-only >current &&
#               test_cmp expect.quoted current
#
#

expecting success of 3902.5 'check fully quoted output from diff-index':

        git diff --name-only HEAD >current &&
        test_cmp expect.quoted current


fatal: ambiguous argument 'HEAD': unknown revision or path not in the
working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
not ok 5 - check fully quoted output from diff-index
#
#
#               git diff --name-only HEAD >current &&
#               test_cmp expect.quoted current
#
#

expecting success of 3902.6 'check fully quoted output from diff-tree':

        git diff --name-only HEAD^ HEAD >current &&
        test_cmp expect.quoted current


fatal: ambiguous argument 'HEAD^': unknown revision or path not in the
working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
not ok 6 - check fully quoted output from diff-tree
#
#
#               git diff --name-only HEAD^ HEAD >current &&
#               test_cmp expect.quoted current
#
#

expecting success of 3902.7 'check fully quoted output from ls-tree':

        git ls-tree --name-only -r HEAD >current &&
        test_cmp expect.quoted current


fatal: Not a valid object name HEAD
not ok 7 - check fully quoted output from ls-tree
#
#
#               git ls-tree --name-only -r HEAD >current &&
#               test_cmp expect.quoted current
#
#

expecting success of 3902.8 'setting core.quotepath':

        git config --bool core.quotepath false


ok 8 - setting core.quotepath

expecting success of 3902.9 'check fully quoted output from ls-files':

        git ls-files >current && test_cmp expect.raw current


--- expect.raw  2020-05-04 19:14:39.000000000 +0000
+++ current     2020-05-04 19:14:40.000000000 +0000
@@ -1,11 +0,0 @@
-Name
-"Name and a\nLF"
-"Name and an\tHT"
-"Name\""
-With SP in it
-"濱野\t純"
-"濱野\n純"
-濱野 純
-"濱野\"純"
-濱野/file
-濱野純
not ok 9 - check fully quoted output from ls-files
#
#
#               git ls-files >current && test_cmp expect.raw current
#
#

expecting success of 3902.10 'check fully quoted output from diff-files':

        git diff --name-only >current &&
        test_cmp expect.raw current


--- expect.raw  2020-05-04 19:14:39.000000000 +0000
+++ current     2020-05-04 19:14:40.000000000 +0000
@@ -1,11 +0,0 @@
-Name
-"Name and a\nLF"
-"Name and an\tHT"
-"Name\""
-With SP in it
-"濱野\t純"
-"濱野\n純"
-濱野 純
-"濱野\"純"
-濱野/file
-濱野純
not ok 10 - check fully quoted output from diff-files
#
#
#               git diff --name-only >current &&
#               test_cmp expect.raw current
#
#

expecting success of 3902.11 'check fully quoted output from diff-index':

        git diff --name-only HEAD >current &&
        test_cmp expect.raw current


fatal: ambiguous argument 'HEAD': unknown revision or path not in the
working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
not ok 11 - check fully quoted output from diff-index
#
#
#               git diff --name-only HEAD >current &&
#               test_cmp expect.raw current
#
#

expecting success of 3902.12 'check fully quoted output from diff-tree':

        git diff --name-only HEAD^ HEAD >current &&
        test_cmp expect.raw current


fatal: ambiguous argument 'HEAD^': unknown revision or path not in the
working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
not ok 12 - check fully quoted output from diff-tree
#
#
#               git diff --name-only HEAD^ HEAD >current &&
#               test_cmp expect.raw current
#
#

expecting success of 3902.13 'check fully quoted output from ls-tree':

        git ls-tree --name-only -r HEAD >current &&
        test_cmp expect.raw current


fatal: Not a valid object name HEAD
not ok 13 - check fully quoted output from ls-tree
#
#
#               git ls-tree --name-only -r HEAD >current &&
#               test_cmp expect.raw current
#
#

# failed 11 among 13 test(s)
1..13

Re: Git 2.26.2 and failed self tests on OS X

[Torsten Bögershausen]

On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
> On Mon, May 4, 2020 at 3:08 PM Torsten Bögershausen <tboegi@web.de> wrote:
> >
> > On Mon, May 04, 2020 at 01:18:06PM -0400, Eric Sunshine wrote:
> > > On Mon, May 4, 2020 at 12:52 PM Carlo Marcelo Arenas Belón
> > > <carenas@gmail.com> wrote:
> > > > On Mon, May 04, 2020 at 03:03:01AM -0400, Jeffrey Walton wrote:
> > > > > I'm building Git 2.26.2 from sources on a Mac-mini with OS X 10.12.6
> > > > > and a PowerMac with OS X 10.5. Some of the t3902-quoted.sh tests are
> > > > > failing.
> > > > >
> > > > > The Mac-Mini uses Bash 3.2.57(1)-release. The PowerMac uses 3.2.17(1)-release.
> > > >
> > > > if the problem is the bash version but you can build/install a newer one that
> > > > doesn't have this problem [...]
> > >
> > > My OS X 10.13.6 has Bash 3.2.57, as well, and I'm unable to reproduce
> > > these failures, so it seems fairly unlikely that the problem Jeffrey
> > > is seeing isn't tied to Bash.
> >
> > Should I read this as
> > ".. seems fairly likely that the problem Jeffrey is seeing is tied to bash" ?
> >
> > I can't reproduce it either, so my reasoning would rather be
> > "The problem is probably not caused by the shipped bash version"
> >
> > It could be something in the environemt, that confuses our tests here.
> >
> > Does
> > ./t3902-quoted.sh   -v -d
> > give anything useful ?
>
> Yes, it looks like the command reveals a couple of problems.
>
> I'll have to look into my iconv recipe. It has not changed in a while,
> so something else may be responsible for the failure.
> https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.
>
> I'm not sure what to do with the HEAD problems.
>
> $ ./t3902-quoted.sh   -v -d
> Initialized empty Git repository in
> /Users/jwalton/Build-Scripts/git-2.26.2/t/trash
> directory.t3902-quoted/.git/
> expecting success of 3902.1 'setup':
>
>         mkdir "$FN" &&
>         for_each_name "echo initial >\"\$name\"" &&
>         git add . &&
>         git commit -q -m Initial &&
>
>         for_each_name "echo second >\"\$name\"" &&
>         git commit -a -m Second &&
>
>         for_each_name "echo modified >\"\$name\""
>
>
> fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
>     precomposed unicode is not supported.
>     If you want to use decomposed unicode, run
>     "git config core.precomposeunicode false"
>

I am quite sure, that the trouble starts here.
It seems as if you are building iconv yourself ?
And that iconv does not include "UTF-8-MAC", which is a speciality
for Mac.

Are there any special reasons to build your own iconv ?
You are the first one repprting this kind of issue,
thanks for reporting it.

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Mon, May 4, 2020 at 4:19 PM Torsten Bögershausen <tboegi@web.de> wrote:
> On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
> > ...
> > > Does
> > > ./t3902-quoted.sh   -v -d
> > > give anything useful ?
> >
> > Yes, it looks like the command reveals a couple of problems.
> >
> > I'll have to look into my iconv recipe. It has not changed in a while,
> > so something else may be responsible for the failure.
> > https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.
> >
> > I'm not sure what to do with the HEAD problems.
> >
> > $ ./t3902-quoted.sh   -v -d
> > Initialized empty Git repository in
> > /Users/jwalton/Build-Scripts/git-2.26.2/t/trash
> > directory.t3902-quoted/.git/
> > expecting success of 3902.1 'setup':
> >
> >         mkdir "$FN" &&
> >         for_each_name "echo initial >\"\$name\"" &&
> >         git add . &&
> >         git commit -q -m Initial &&
> >
> >         for_each_name "echo second >\"\$name\"" &&
> >         git commit -a -m Second &&
> >
> >         for_each_name "echo modified >\"\$name\""
> >
> >
> > fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
> >     precomposed unicode is not supported.
> >     If you want to use decomposed unicode, run
> >     "git config core.precomposeunicode false"
> >
>
> I am quite sure, that the trouble starts here.
> It seems as if you are building iconv yourself ?
> And that iconv does not include "UTF-8-MAC", which is a speciality
> for Mac.

Oh, thanks. I was not aware.

I'll open a bug report with the iconv folks letting them know
configure has some gaps on OS X.

> Are there any special reasons to build your own iconv ?

Yes, iconv bugs on older platforms.

The only way I've found to avoid intermittent problems on older
platforms is using the latest iconv (which is pretty stable nowadays),
and avoid the glibc iconv.

Jeff

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Mon, May 4, 2020 at 4:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
>
> On Mon, May 4, 2020 at 4:19 PM Torsten Bögershausen <tboegi@web.de> wrote:
> > On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
> > > ...
> > > > Does
> > > > ./t3902-quoted.sh   -v -d
> > > > give anything useful ?
> > >
> > > Yes, it looks like the command reveals a couple of problems.
> > >
> > > I'll have to look into my iconv recipe. It has not changed in a while,
> > > so something else may be responsible for the failure.
> > > https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.
> > >
> > > I'm not sure what to do with the HEAD problems.
> > >
> > > $ ./t3902-quoted.sh   -v -d
> > > Initialized empty Git repository in
> > > /Users/jwalton/Build-Scripts/git-2.26.2/t/trash
> > > directory.t3902-quoted/.git/
> > > expecting success of 3902.1 'setup':
> > >
> > >         mkdir "$FN" &&
> > >         for_each_name "echo initial >\"\$name\"" &&
> > >         git add . &&
> > >         git commit -q -m Initial &&
> > >
> > >         for_each_name "echo second >\"\$name\"" &&
> > >         git commit -a -m Second &&
> > >
> > >         for_each_name "echo modified >\"\$name\""
> > >
> > >
> > > fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
> > >     precomposed unicode is not supported.
> > >     If you want to use decomposed unicode, run
> > >     "git config core.precomposeunicode false"
> > >
> >
> > I am quite sure, that the trouble starts here.
> > It seems as if you are building iconv yourself ?
> > And that iconv does not include "UTF-8-MAC", which is a specialty
> > for Mac.
>
> Oh, thanks. I was not aware.
>
> I'll open a bug report with the iconv folks letting them know
> configure has some gaps on OS X.

Thanks again Torsten.

I am not seeing an option to enable the encoding in iConv. Would you
happen to know what is needed?

Thanks again.

$ cd libiconv-1.16
$ ./configure --help
`configure' configures libiconv 1.16 to adapt to many kinds of systems.

Usage: ./configure [OPTION]... [VAR=VALUE]...

To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE.  See below for descriptions of some of the useful variables.

Defaults for the options are specified in brackets.

Configuration:
  -h, --help              display this help and exit
      --help=short        display options specific to this package
      --help=recursive    display the short help of all the included packages
  -V, --version           display version information and exit
  -q, --quiet, --silent   do not print `checking ...' messages
      --cache-file=FILE   cache test results in FILE [disabled]
  -C, --config-cache      alias for `--cache-file=config.cache'
  -n, --no-create         do not create output files
      --srcdir=DIR        find the sources in DIR [configure dir or `..']

Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]

By default, `make install' will install all the files in
`/usr/local/bin', `/usr/local/lib' etc.  You can specify
an installation prefix other than `/usr/local' using `--prefix',
for instance `--prefix=$HOME'.

For better control, use the options below.

Fine tuning of the installation directories:
  --bindir=DIR            user executables [EPREFIX/bin]
  --sbindir=DIR           system admin executables [EPREFIX/sbin]
  --libexecdir=DIR        program executables [EPREFIX/libexec]
  --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
  --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
  --libdir=DIR            object code libraries [EPREFIX/lib]
  --includedir=DIR        C header files [PREFIX/include]
  --oldincludedir=DIR     C header files for non-gcc [/usr/include]
  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
  --datadir=DIR           read-only architecture-independent data [DATAROOTDIR]
  --infodir=DIR           info documentation [DATAROOTDIR/info]
  --localedir=DIR         locale-dependent data [DATAROOTDIR/locale]
  --mandir=DIR            man documentation [DATAROOTDIR/man]
  --docdir=DIR            documentation root [DATAROOTDIR/doc/libiconv]
  --htmldir=DIR           html documentation [DOCDIR]
  --dvidir=DIR            dvi documentation [DOCDIR]
  --pdfdir=DIR            pdf documentation [DOCDIR]
  --psdir=DIR             ps documentation [DOCDIR]

Program names:
  --program-prefix=PREFIX            prepend PREFIX to installed program names
  --program-suffix=SUFFIX            append SUFFIX to installed program names
  --program-transform-name=PROGRAM   run sed PROGRAM on installed program names

System types:
  --build=BUILD     configure for building on BUILD [guessed]
  --host=HOST       cross-compile to build programs to run on HOST [BUILD]

Optional Features:
  --disable-option-checking  ignore unrecognized --enable/--with options
  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
  --enable-silent-rules   less verbose build output (undo: "make V=1")
  --disable-silent-rules  verbose build output (undo: "make V=0")
  --enable-dependency-tracking
                          do not reject slow dependency extractors
  --disable-dependency-tracking
                          speeds up one-time build
  --disable-largefile     omit support for large files
  --enable-static[=PKGS]  build static libraries [default=no]
  --enable-shared[=PKGS]  build shared libraries [default=yes]
  --enable-fast-install[=PKGS]
                          optimize for fast installation [default=yes]
  --disable-libtool-lock  avoid locking (might break parallel builds)
  --enable-relocatable    install a package that can be moved in the file
                          system
  --enable-extra-encodings
                          add support for a few rarely used encodings
  --disable-rpath         do not hardcode runtime library paths
  --disable-nls           do not use Native Language Support

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-pic[=PKGS]       try to use only PIC/non-PIC objects [default=use
                          both]
  --with-aix-soname=aix|svr4|both
                          shared library versioning (aka "SONAME") variant to
                          provide on AIX, [default=aix].
  --with-gnu-ld           assume the C compiler uses GNU ld [default=no]
  --with-sysroot[=DIR]    Search for dependent libraries within DIR (or the
                          compiler's sysroot if not specified).
  --with-gnu-ld           assume the C compiler uses GNU ld [default=no]
  --with-libiconv-prefix[=DIR]  search for libiconv in DIR/include and DIR/lib
  --without-libiconv-prefix     don't search for libiconv in
includedir and libdir
  --with-libintl-prefix[=DIR]  search for libintl in DIR/include and DIR/lib
  --without-libintl-prefix     don't search for libintl in includedir and libdir

Some influential environment variables:
  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  LIBS        libraries to pass to the linker, e.g. -l<library>
  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
              you have headers in a nonstandard directory <include dir>
  CPP         C preprocessor
  LT_SYS_LIBRARY_PATH
              User-defined run-time library search path.

Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.

Re: Git 2.26.2 and failed self tests on OS X

[Torsten Bögershausen]

On Mon, May 04, 2020 at 05:39:50PM -0400, Jeffrey Walton wrote:
> On Mon, May 4, 2020 at 4:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
> >
> > On Mon, May 4, 2020 at 4:19 PM Torsten Bögershausen <tboegi@web.de> wrote:
> > > On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
> > > > ...
> > > > > Does
> > > > > ./t3902-quoted.sh   -v -d
> > > > > give anything useful ?
> > > >
> > > > Yes, it looks like the command reveals a couple of problems.
> > > >
> > > > I'll have to look into my iconv recipe. It has not changed in a while,
> > > > so something else may be responsible for the failure.
> > > > https://github.com/noloader/Build-Scripts/blob/master/build-iconv.sh.
> > > >
> > > > I'm not sure what to do with the HEAD problems.
> > > >
> > > > $ ./t3902-quoted.sh   -v -d
> > > > Initialized empty Git repository in
> > > > /Users/jwalton/Build-Scripts/git-2.26.2/t/trash
> > > > directory.t3902-quoted/.git/
> > > > expecting success of 3902.1 'setup':
> > > >
> > > >         mkdir "$FN" &&
> > > >         for_each_name "echo initial >\"\$name\"" &&
> > > >         git add . &&
> > > >         git commit -q -m Initial &&
> > > >
> > > >         for_each_name "echo second >\"\$name\"" &&
> > > >         git commit -a -m Second &&
> > > >
> > > >         for_each_name "echo modified >\"\$name\""
> > > >
> > > >
> > > > fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
> > > >     precomposed unicode is not supported.
> > > >     If you want to use decomposed unicode, run
> > > >     "git config core.precomposeunicode false"
> > > >
> > >
> > > I am quite sure, that the trouble starts here.
> > > It seems as if you are building iconv yourself ?
> > > And that iconv does not include "UTF-8-MAC", which is a specialty
> > > for Mac.
> >
> > Oh, thanks. I was not aware.
> >
> > I'll open a bug report with the iconv folks letting them know
> > configure has some gaps on OS X.
>
> Thanks again Torsten.
>
> I am not seeing an option to enable the encoding in iConv. Would you
> happen to know what is needed?
>
> Thanks again.
>
> $ cd libiconv-1.16
> $ ./configure --help
> `configure' configures libiconv 1.16 to adapt to many kinds of systems.

Hej Jeffrey,
I am not an iconv expert - and don't intend to become one.
The UTF-8-MAC conversion does probably only exist in the
code for iconv shipped on Mac OS - so it could happen that this
code is not open source.
There may be ways to "extract it" :
Either that information is part of the unicode specification,
and there may be tables that can be converted into c-code that
fits into the iconv code base. I don't know.
Or you can use the iconv on MacOs itself to generate such a table.
That is all somewhat behind my horizon.

For Git (under MacOS) we need UTF-8-MAC to compensate for HPS,
(or HPF+) so that all filenames with non-ASCII codepoints are
stored in the repo in precomposed unicode.
This is what Linux and Windoes and Unix is using, this commit
may give more information about the what and why:

  commit 76759c7dff53e8c84e975b88cb8245587c14c7ba
  Author: Torsten Bögershausen <tboegi@web.de>
  Date:   Sun Jul 8 15:50:25 2012 +0200
    git on Mac OS and precomposed unicode

By the way, does
./t3910-mac-os-precompose.sh
work on your system ?

So far I am not aware about any problems with iconv from MacOs
being reported here on the list.
Which problems/bugs did you found in the iconv shipped with MacOs,
especially together with Git ?

[snip]

Re: Git 2.26.2 and failed self tests on OS X

[Philippe Blain]

Hi everyone,

> Le 5 mai 2020 à 00:10, Torsten Bögershausen <tboegi@web.de> a écrit :
> 
> On Mon, May 04, 2020 at 05:39:50PM -0400, Jeffrey Walton wrote:
>> On Mon, May 4, 2020 at 4:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
>>> 
>>> On Mon, May 4, 2020 at 4:19 PM Torsten Bögershausen <tboegi@web.de> wrote:
>>>> On Mon, May 04, 2020 at 03:20:11PM -0400, Jeffrey Walton wrote:
>>>>> ...
>>>>>> Does
>>>>>> ./t3902-quoted.sh   -v -d
>>>>>> give anything useful ?
>>>>> 
>>>>> Yes, it looks like the command reveals a couple of problems.
>>>>> 
>>>>> fatal: iconv_open(UTF-8,UTF-8-MAC) failed, but needed:
>>>>>    precomposed unicode is not supported.
>>>>>    If you want to use decomposed unicode, run
>>>>>    "git config core.precomposeunicode false"

I also came across this error a few months ago with the 
conda-forge build of Git [1], which for the moment builds against their own
build of (GNU) iconv.

>>>> 
>>>> I am quite sure, that the trouble starts here.
>>>> It seems as if you are building iconv yourself ?
>>>> And that iconv does not include "UTF-8-MAC", which is a specialty
>>>> for Mac.
>>> 
>>> Oh, thanks. I was not aware.
>>> 
>>> I'll open a bug report with the iconv folks letting them know
>>> configure has some gaps on OS X.
>> 
>> Thanks again Torsten.
>> 
>> I am not seeing an option to enable the encoding in iConv. Would you
>> happen to know what is needed?
>> 
>> Thanks again.
>> 
>> $ cd libiconv-1.16
>> $ ./configure --help
>> `configure' configures libiconv 1.16 to adapt to many kinds of systems.
> 
> Hej Jeffrey,
> I am not an iconv expert - and don't intend to become one.
> The UTF-8-MAC conversion does probably only exist in the
> code for iconv shipped on Mac OS - so it could happen that this
> code is not open source.

Yes, this encoding only exists in the Apple fork of iconv,
which is open-source (probably because of the original 
license); tarballs can be found at [2]. However in typical Apple fashion
the versioning of their iconv fork is kind of undecipherable and hard to relate to the
upstream iconv versioning. 

Also, I searched the iconv mailing list [3] when I came across this error and the consensus
seemed to be that they do not intend to add this encoding upstream.

Cheers,

Philippe.

[1] https://github.com/conda-forge/git-feedstock/issues/50
[2] https://opensource.apple.com/tarballs/libiconv/
[3] https://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=utf-8-mac&submit=Search%21&idxname=bug-gnu-libiconv&max=20&result=normal&sort=score

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Tue, May 5, 2020 at 8:26 AM Philippe Blain <philippe.blain@me.com> wrote:
>  ...
> > I am not an iconv expert - and don't intend to become one.
> > The UTF-8-MAC conversion does probably only exist in the
> > code for iconv shipped on Mac OS - so it could happen that this
> > code is not open source.
>
> Yes, this encoding only exists in the Apple fork of iconv,
> which is open-source (probably because of the original
> license); tarballs can be found at [2]. However in typical Apple fashion
> the versioning of their iconv fork is kind of undecipherable and hard to relate to the
> upstream iconv versioning.
>
> Also, I searched the iconv mailing list [3] when I came across this error and the consensus
> seemed to be that they do not intend to add this encoding upstream.
>
> Cheers,
>
> Philippe.
>
> [1] https://github.com/conda-forge/git-feedstock/issues/50
> [2] https://opensource.apple.com/tarballs/libiconv/
> [3] https://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=utf-8-mac&submit=Search%21&idxname=bug-gnu-libiconv&max=20&result=normal&sort=score

Thanks Phillipe.

I found two sources for libiconv with UTF-8-Mac. First is a GitHub at
https://github.com/fumiyas/libiconv-utf8mac. The second is the
opensource.apple.com link you provided at [2].

The GitHub claims to be layered on top of libiconv 1.16, but I was not
able to get a working tarball from it. It looks like Apple's
libiconv-59.tar.gz is the answer to this problem.

Thanks again for the help.

Jeff

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Tue, May 5, 2020 at 12:10 AM Torsten Bögershausen <tboegi@web.de> wrote:
>
> On Mon, May 04, 2020 at 05:39:50PM -0400, Jeffrey Walton wrote:
> > On Mon, May 4, 2020 at 4:30 PM Jeffrey Walton <noloader@gmail.com> wrote:
> > >
> > > ...
> By the way, does
> ./t3910-mac-os-precompose.sh
> work on your system ?
>
> So far I am not aware about any problems with iconv from MacOs
> being reported here on the list.
> Which problems/bugs did you found in the iconv shipped with MacOs,
> especially together with Git ?

Thanks again Torsten .

So there are several problems with libiconv. First, sometimes it is
too old and carries bugs with it. For example, sed and awk fail to
build and pass their self tests because they expect a minimum version
of libiconv.

Second, there are some modern bugs that needs fixing. They were
reported after libiconv 1.16 was released. It needs patches.

Third, libiconv cannot get through testing with UBsan. It causes a
number of packages to fail to configure or pass their self tests. It
needs patches.

Fourth, libiconv cannot get through testing with Asan and Valgrind. It
leaks resources like a sieve, and leaves memory and file handles open.
It causes a number of packages to fail to configure or pass their self
tests. It needs more patches.

Due to all the problems with libiconv I just patch it and build it
everywhere I need it.

The other option is to wait for the maintainers to fix libiconv. Based
on some reading about the UTF-8-Mac problem I don't think that will
happen. And they are not interested in fixing the resource leaks. GNU
has such low coding standards...

Jeff

Re: Git 2.26.2 and failed self tests on OS X

[Jeffrey Walton]

On Wed, May 6, 2020 at 4:36 PM Jeffrey Walton <noloader@gmail.com> wrote:
>
> On Tue, May 5, 2020 at 8:26 AM Philippe Blain <philippe.blain@me.com> wrote:
> >  ...
> > > I am not an iconv expert - and don't intend to become one.
> > > The UTF-8-MAC conversion does probably only exist in the
> > > code for iconv shipped on Mac OS - so it could happen that this
> > > code is not open source.
> >
> > Yes, this encoding only exists in the Apple fork of iconv,
> > which is open-source (probably because of the original
> > license); tarballs can be found at [2]. However in typical Apple fashion
> > the versioning of their iconv fork is kind of undecipherable and hard to relate to the
> > upstream iconv versioning.
> >
> > Also, I searched the iconv mailing list [3] when I came across this error and the consensus
> > seemed to be that they do not intend to add this encoding upstream.
> >
> > [1] https://github.com/conda-forge/git-feedstock/issues/50
> > [2] https://opensource.apple.com/tarballs/libiconv/
> > [3] https://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=utf-8-mac&submit=Search%21&idxname=bug-gnu-libiconv&max=20&result=normal&sort=score
>
> Thanks Phillipe.
>
> I found two sources for libiconv with UTF-8-Mac. First is a GitHub at
> https://github.com/fumiyas/libiconv-utf8mac. The second is the
> opensource.apple.com link you provided at [2].
>
> The GitHub claims to be layered on top of libiconv 1.16, but I was not
> able to get a working tarball from it. It looks like Apple's
> libiconv-59.tar.gz is the answer to this problem.

I had to punt on Apple's port. libiconv-59.tar.gz is libiconv-1.11. It
is too old to be useful.

libiconv-utf8mac is based on libiconv-1.16 and worked fine. It took my
patches after I figured out the complete procedure. On the upside,
libiconv-utf8mac works on Linux and Solaris too. So there is support
for UTF-8-Mac if needed.

Jeff