[Bug report] Git incorrectly selects language in macos

[Bug report] Git incorrectly selects language in macos

[Niko Dzhus]

It doesn't use English when other language is available as a secondary language.

Reproducing:

1. Open "Language & Region" in macos settings
2. In "Preferred languages" box, set English as a primary language.
3. Add another language, that git is translated to, as a secondary
language, for example, French or German.
4. Run any git command - git will use the secondary language, instead
of English.

When the secondary language is removed, then git starts using English again.

I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

Re: [Bug report] Git incorrectly selects language in macos

[Ævar Arnfjörð Bjarmason]

On Fri, Sep 14 2018, Niko Dzhus wrote:

> It doesn't use English when other language is available as a secondary language.
>
> Reproducing:
>
> 1. Open "Language & Region" in macos settings
> 2. In "Preferred languages" box, set English as a primary language.
> 3. Add another language, that git is translated to, as a secondary
> language, for example, French or German.
> 4. Run any git command - git will use the secondary language, instead
> of English.
>
> When the secondary language is removed, then git starts using English again.
>
> I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

What's the output of these two commands for you:

 1. locale
 2. env | grep -e LC -e LANG

We don't do any such magic ourselves, so whatever this is is down to how
i18n in general works on your system, do you have any other translated
command-line program that works differently?

I suspect there's some DWYM logic here that always treats English as a
secondary language.

Do you also e.g. get the same results if you select say Swedish as a
primary language and German as a secondary? I.e. a Git in German, as
opposed to Swedish?

Re: [Bug report] Git incorrectly selects language in macos

[Niko Dzhus]

Tried what you suggested - it seems, it only ignores English. In you
example, with Swedish as primary and German as secondary, git uses
Swedish.

With more that one secondary language, the one with a higher priority
is being used, as expected. I also tried using non-generic English
(English-UK and English-US), but they also get ignored.

Terminal commands return the following:

➜  ~ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
➜  ~ env | grep -e LC -e LANG
LC_CTYPE=UTF-8
➜  ~

It doesn't change with primary/secondary language switching. I don't
have any manual overrides in my .zshrc and .zprofile for those
neither.

On Sat, Sep 15, 2018 at 12:57 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Fri, Sep 14 2018, Niko Dzhus wrote:
>
> > It doesn't use English when other language is available as a secondary language.
> >
> > Reproducing:
> >
> > 1. Open "Language & Region" in macos settings
> > 2. In "Preferred languages" box, set English as a primary language.
> > 3. Add another language, that git is translated to, as a secondary
> > language, for example, French or German.
> > 4. Run any git command - git will use the secondary language, instead
> > of English.
> >
> > When the secondary language is removed, then git starts using English again.
> >
> > I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .
>
> What's the output of these two commands for you:
>
>  1. locale
>  2. env | grep -e LC -e LANG
>
> We don't do any such magic ourselves, so whatever this is is down to how
> i18n in general works on your system, do you have any other translated
> command-line program that works differently?
>
> I suspect there's some DWYM logic here that always treats English as a
> secondary language.
>
> Do you also e.g. get the same results if you select say Swedish as a
> primary language and German as a secondary? I.e. a Git in German, as
> opposed to Swedish?

Re: [Bug report] Git incorrectly selects language in macos

[Niko Dzhus]

Looks like the issue appeared after updating git from brew.

I tried it on a different mac laptop, git 2.18 still used English, but
after updating to 2.19 it started using secondary language.

A quick search revealed that brew changed how it builds git recently.
I think, it just didn't include i18n by default before, so I never
noticed this.
Here's the history of formula changes:
https://github.com/Homebrew/homebrew-core/commits/master/Formula/git.rb
Also, I found this thread https://github.com/Homebrew/homebrew-core/issues/31980

Anybody here familiar enough with the build process and dependencies
of git to pinpoint what exactly is causing this and how to fix it?...

On Fri, Sep 14, 2018 at 10:08 PM Niko Dzhus <primenico@gmail.com> wrote:
>
> It doesn't use English when other language is available as a secondary language.
>
> Reproducing:
>
> 1. Open "Language & Region" in macos settings
> 2. In "Preferred languages" box, set English as a primary language.
> 3. Add another language, that git is translated to, as a secondary
> language, for example, French or German.
> 4. Run any git command - git will use the secondary language, instead
> of English.
>
> When the secondary language is removed, then git starts using English again.
>
> I have git 2.19.0, installed from brew, and my OS is macOS 10.13.6 .

Re: [Bug report] Git incorrectly selects language in macos

[Eric Sunshine]

On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
> Looks like the issue appeared after updating git from brew.
>
> A quick search revealed that brew changed how it builds git recently.
> I think, it just didn't include i18n by default before, so I never
> noticed this.
>
> Anybody here familiar enough with the build process and dependencies
> of git to pinpoint what exactly is causing this and how to fix it?...

This problem is not specific to Git. Earlier in the thread, Ævar
asked[1] if the problem also occurs with other command-line programs,
and indeed it does. For instance, I tried with 'wget' installed via
brew, and it exhibits the same odd behavior. Ævar suggested that there
might be some magic special-casing English, which makes me wonder if
brew builds such magic into gettext(?) or if the magic is part of
MacOS itself.

[1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/

Re: [Bug report] Git incorrectly selects language in macos

[Nate Weaver]

> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
> 
> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>> Looks like the issue appeared after updating git from brew.
>> 
>> A quick search revealed that brew changed how it builds git recently.
>> I think, it just didn't include i18n by default before, so I never
>> noticed this.
>> 
>> Anybody here familiar enough with the build process and dependencies
>> of git to pinpoint what exactly is causing this and how to fix it?...
> 
> This problem is not specific to Git. Earlier in the thread, Ævar
> asked[1] if the problem also occurs with other command-line programs,
> and indeed it does. For instance, I tried with 'wget' installed via
> brew, and it exhibits the same odd behavior. Ævar suggested that there
> might be some magic special-casing English, which makes me wonder if
> brew builds such magic into gettext(?) or if the magic is part of
> MacOS itself.
> 
> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/

I discovered that moving/renaming /usr/local/share/locale made this
issue go away for me. While 'de' and 'fr' have a git.mo file in their
LC_MESSAGES subdirectory, there is no 'en' directory.

If I copy the 'fr' version to a newly-created 'en' directory, then all my
git output is in French; my language order in System Preferences is
English (US) -> German -> Japanese.

Not sure if this is gettext's or macOS's issue.

--Nate Weaver (Wevah)

Re: [Bug report] Git incorrectly selects language in macos

[Nate Weaver]

> On Jan 15, 2019, at 09:35:52, Nate Weaver <wevah@derailer.org> wrote:
> 
>> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
>> 
>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>>> Looks like the issue appeared after updating git from brew.
>>> 
>>> A quick search revealed that brew changed how it builds git recently.
>>> I think, it just didn't include i18n by default before, so I never
>>> noticed this.
>>> 
>>> Anybody here familiar enough with the build process and dependencies
>>> of git to pinpoint what exactly is causing this and how to fix it?...
>> 
>> This problem is not specific to Git. Earlier in the thread, Ævar
>> asked[1] if the problem also occurs with other command-line programs,
>> and indeed it does. For instance, I tried with 'wget' installed via
>> brew, and it exhibits the same odd behavior. Ævar suggested that there
>> might be some magic special-casing English, which makes me wonder if
>> brew builds such magic into gettext(?) or if the magic is part of
>> MacOS itself.
>> 
>> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/
> 
> I discovered that moving/renaming /usr/local/share/locale made this
> issue go away for me. While 'de' and 'fr' have a git.mo file in their
> LC_MESSAGES subdirectory, there is no 'en' directory.
> 
> If I copy the 'fr' version to a newly-created 'en' directory, then all my
> git output is in French; my language order in System Preferences is
> English (US) -> German -> Japanese.
> 
> Not sure if this is gettext's or macOS's issue.
> 
> --Nate Weaver (Wevah)

Upon further digging, this is an issue in gettext's code on macOS:
The function _nl_language_preferences_default (in langprefs.c) specifically
breaks early when it sees the literal string "en" in the list (from the
"AppleLanguages" defaults key), but not when it gets "en-US", etc.

I.e., it doesn't check for a prefix.

This can be fixed (though there might be a better way) simply by replacing
both instances of

if (strcmp (buf, "en") == 0)

with

if (strncmp (buf, "en", 2) == 0)

in gettext's langprefs.c.

Re: [Bug report] Git incorrectly selects language in macos

[Nate Weaver]

--Nate Weaver (Wevah)

> On Jan 15, 2019, at 10:43:48, Nate Weaver <wevah@derailer.org> wrote:
> 
>> On Jan 15, 2019, at 09:35:52, Nate Weaver <wevah@derailer.org> wrote:
>> 
>>> On Sep 16, 2018, at 02:15:33, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>> 
>>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:
>>>> Looks like the issue appeared after updating git from brew.
>>>> 
>>>> A quick search revealed that brew changed how it builds git recently.
>>>> I think, it just didn't include i18n by default before, so I never
>>>> noticed this.
>>>> 
>>>> Anybody here familiar enough with the build process and dependencies
>>>> of git to pinpoint what exactly is causing this and how to fix it?...
>>> 
>>> This problem is not specific to Git. Earlier in the thread, Ævar
>>> asked[1] if the problem also occurs with other command-line programs,
>>> and indeed it does. For instance, I tried with 'wget' installed via
>>> brew, and it exhibits the same odd behavior. Ævar suggested that there
>>> might be some magic special-casing English, which makes me wonder if
>>> brew builds such magic into gettext(?) or if the magic is part of
>>> MacOS itself.
>>> 
>>> [1]: https://public-inbox.org/git/87a7ojlp31.fsf@evledraar.gmail.com/
>> 
>> I discovered that moving/renaming /usr/local/share/locale made this
>> issue go away for me. While 'de' and 'fr' have a git.mo file in their
>> LC_MESSAGES subdirectory, there is no 'en' directory.
>> 
>> If I copy the 'fr' version to a newly-created 'en' directory, then all my
>> git output is in French; my language order in System Preferences is
>> English (US) -> German -> Japanese.
>> 
>> Not sure if this is gettext's or macOS's issue.
>> 
>> --Nate Weaver (Wevah)
> 
> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.
> 
> I.e., it doesn't check for a prefix.
> 
> This can be fixed (though there might be a better way) simply by replacing
> both instances of
> 
> if (strcmp (buf, "en") == 0)
> 
> with
> 
> if (strncmp (buf, "en", 2) == 0)
> 
> in gettext's langprefs.c.

Nevermind: I've been informed that this has already been fixed on the git trunk
for gettext, and will be in the next release.

(See: https://savannah.gnu.org/bugs/?49560)

Re: [Bug report] Git incorrectly selects language in macos

[Eric Sunshine]

On Tue, Jan 15, 2019 at 2:06 PM Nate Weaver <wevah@derailer.org> wrote:
> Nevermind: I've been informed that this has already been fixed on the git trunk
> for gettext, and will be in the next release.
>
> (See: https://savannah.gnu.org/bugs/?49560)

Thanks for the follow-up report. It's good to know that this is fixed
at the correct location upstream.

I tried updating the Homebrew bug report[1] regarding this problem
with respect to Git in order to add the new information you provided,
but the ticket has been "locked" so no additional comments can be made
there, which is unfortunate.

[1]: https://github.com/Homebrew/homebrew-core/issues/31980

Re: [Bug report] Git incorrectly selects language in macos

[Jonathan Nieder]

Nate Weaver wrote:
>>> On Fri, Sep 14, 2018 at 10:20 PM Niko Dzhus <primenico@gmail.com> wrote:

>>>> Looks like the issue appeared after updating git from brew.
[...]
>>>> A quick search revealed that brew changed how it builds git recently.
>>>> I think, it just didn't include i18n by default before, so I never
>>>> noticed this.
[...]
> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.
>
> I.e., it doesn't check for a prefix.
>
> This can be fixed (though there might be a better way) simply by replacing
> both instances of
>
> if (strcmp (buf, "en") == 0)
>
> with
>
> if (strncmp (buf, "en", 2) == 0)
>
> in gettext's langprefs.c.

Nice sleuthing.  Do you have a bug tracking this in homebrew's issue
tracker, either for the git formula or the gettext formula?

Thanks,
Jonathan

Re: [Bug report] Git incorrectly selects language in macos

[Jonathan Nieder]

Nate Weaver wrote:

> Upon further digging, this is an issue in gettext's code on macOS:
> The function _nl_language_preferences_default (in langprefs.c) specifically
> breaks early when it sees the literal string "en" in the list (from the
> "AppleLanguages" defaults key), but not when it gets "en-US", etc.

Sorry I missed your other followup.

As you mentioned, the fix is at https://savannah.gnu.org/bugs/?49560.
There isn't a gettext release yet with that fix, so perhaps
https://github.com/Homebrew/homebrew-core/blob/master/Formula/gettext.rb
could be updated to apply that fix as a patch
(https://docs.brew.sh/Formula-Cookbook#patches).

Thanks again and sorry for the noise,
Jonathan