Re: [PATCH/RFC v7 1/2] Add infrastructure for translating Git with gettext

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Sat, Jun 5, 2010 at 13:57, Jakub Narebski <jnareb@gmail.com> wrote:
> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> All of the interface messages in Git core are currently hardcoded in
>> English. Change that by optionally enabling translation of the core C,
>> Shell and Perl programs via GNU gettext. If you set the appropriate
>> LC_* variables Git will speak your language, provided that someone has
>> submitted a translation.
> [...]
>
>> Implementation and usage notes:
> [...]
>>  * Perl:
>>
>>    Perl code that wants to be localized should use the new Git::I18n
>>    module. It imports a __ function into the caller's package by
>>    default.
>>
>>    Instead of using the high level Locale::TextDomain interface I've
>>    opted to use the low-level (equivalent to the C interface)
>>    Locale::Messages module, which Locale::TextDomain itself uses.
>>
>>    Locale::TextDomain does a lot of redundant work we don't need, and
>>    some of it would potentially introduce bugs. It tries to set the
>>    $TEXTDOMAIN based on package of the caller, and has its own
>>    hardcoded paths where it'll search for messages.
>
> Actually both of those can be set using Locale::TextDomain->import()
> call.  See e.g. this answer on StackOverflow:
>  http://stackoverflow.com/questions/2965626/examples-of-localization-in-perl-using-gettext-and-localetextdomain-with-fallb/2967872#2967872
>
>>    [...] In any case, this is an issue wholly internal
>>    Git::I18N. Its guts can be changed later if that's deemed
>>    necessary.
>
> Right.

Locale::TextDomain's interface simply does more than
Locale::Messages. And since I'm trying to present exactly the same
interface to C, Shell and Perl any code that uses it has to spend
effort on disabling its features.

E.g.

  * No I don't want a tied hash (yes, I know I don't have to export it
    in turn):

        BEGIN {
            # Tie the hash to gettext().
            tie %__, '__TiedTextDomain', \&__tied_gettext;
            $__ = \%__;

  * I *only* want our TEXTDOMAINDIR, not this. And there's no way to
    turn it off:

    	# Add default search directories, but only if they exist.
    	for my $dir (qw (/usr/share/locale /usr/local/share/locale)) {
            if (-d $dir) {
                @default_dirs = ($dir);

  * I also don't want to search for locale files in Perl's @INC, or
    $textdomaindir/LocaleData:

        unless (exists $bound_dirs{$textdomain}) {
    		@search_dirs = map $_ . '/LocaleData', @INC, @default_dirs
    			unless @search_dirs;
    		$bound_dirs{$textdomain} = [@search_dirs];

  * Here's what the normal functions look like. Since caller() will be
    the top-level caller and not Git::I18N (unless I define a wrapper
    function, which Git::I18N doesn't do) I'd have to take measures to
    set the textdomain for the importer's package as well as my own:

        # Normal gettext.
        sub __ ($)
        {
            my $msgid = shift;
            my $package = caller;

     And that sub does this:

        __find_domain $textdomain if
    		defined $textdomain && defined $bound_dirs{$textdomain};

  * Which again bypasses what I just want to do with bindtextdomain():

        sub __find_domain ($)
        {
        	my $domain = shift;
        	
        	my $try_dirs = $bound_dirs{$domain};
        	
        	if (defined $try_dirs) {
        		my $found_dir = '';
        		
        		TRYDIR: foreach my $dir (map { abs_path $_ } grep { -d $_ }
@$try_dirs) {
        			# Is there a message catalog?  We have to search recursively
        			# for it.  Since globbing is reported to be buggy under
        			# MS-DOS, we roll our own version.
        			local *DIR;
        			if (opendir DIR, $dir) {
        				my @files = map { "$dir/$_/LC_MESSAGES/$domain.mo" }
        					grep { ! /^\.\.?$/ } readdir DIR;

        				foreach my $file (@files) {
        					if (-f $file || -l $file) {
        						# If we find a non-readable file on our way,
        						# we access has been disabled on purpose.
        						# Therefore no -r check here.
        						$found_dir = $dir;
        						last TRYDIR;
        					}
        				}
        			}
        		}
        		
        		# If there was no success, this will fall back to the default search
        		# directories.
        		bindtextdomain $domain => $found_dir;

  * That'll only happen on the first invocation though, later it'll
    just skip that and do:

        dgettext $textdomain => $msgid;

None of this is a showstopper. It's just more code to test, override
and stuff that can go wrong.

It doesn't make any sense to use it when all I want is simply
bindtextdomain(git => $dir) && gettext($msgid).

>> --- a/INSTALL
>> +++ b/INSTALL
>> @@ -93,6 +93,14 @@ Issues of note:
>>         history graphically, and in git-gui.  If you don't want gitk or
>>         git-gui, you can use NO_TCLTK.
>>
>> +     - The GNU "libintl" library is used by default for localizing
>> +       Git. It needs a gettext.h on the system for C code, gettext.sh
>> +       for shell scripts, and libintl-perl for Perl programs.
>> +
>> +       Set NO_GETTEXT to disable localization support and make Git only
>> +       use English. Under autoconf the configure script will do this
>> +       automatically if it can't find libintl on the system.
>> +
>
> Shouldn't you also add here that you need also "libintl-perl" to have
> those commands that are written in Perl localized?

I pondered it, it's it's documented in the INSTALL file, but all the
comments in the Makefile are pertinent only to compiling the C code.

Since libintl-perl isn't a compilation requirement, or anything you
need to worry about when running make I didn't add it in the Makefile
comments.

>> diff --git a/Makefile b/Makefile
>> index d5d6565..3040000 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -28,6 +28,15 @@ all::
>>  # Define NO_EXPAT if you do not have expat installed.  git-http-push is
>>  # not built, and you cannot push using http:// and https:// transports.
>>  #
>> +# Define NO_GETTEXT if you don't want to build with Git with gettext
>> +# support. Building it requires GNU libintl, and additionally
>> +# libintl-perl at runtime.
>> +#
>> +# Define NEEDS_LIBINTL if you haven't set NO_GETTEXT and your system
>> +# needs to be explicitly linked to -lintl. It's defined automatically
>> +# on platforms where we don't expect glibc (Linux, Hurd,
>> +# GNU/kFreeBSD), which includes libintl.
> [...]
>
>> diff --git a/config.mak.in b/config.mak.in
>> index 0d4b64d..a15f3c1 100644
>> --- a/config.mak.in
>> +++ b/config.mak.in
>> @@ -32,6 +32,7 @@ NO_CURL=@NO_CURL@
>>  NO_EXPAT=@NO_EXPAT@
>>  NO_LIBGEN_H=@NO_LIBGEN_H@
>>  HAVE_PATHS_H=@HAVE_PATHS_H@
>> +NO_GETTEXT=@NO_GETTEXT@
>
> No
>
>  +NEEDS_LIBINTL=@NEEDS_LIBINTL@
>
> (see also below)?
>
>> diff --git a/configure.ac b/configure.ac
>> index 71038fc..7bebfd8 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -730,6 +730,12 @@ AC_CHECK_HEADER([paths.h],
>>  [HAVE_PATHS_H=])
>>  AC_SUBST(HAVE_PATHS_H)
>>  #
>> +# Define NO_GETTEXT if you don't have libintl.h
>> +AC_CHECK_HEADER([libintl.h],
>> +[NO_GETTEXT=],
>> +[NO_GETTEXT=YesPlease])
>> +AC_SUBST(NO_GETTEXT)
>> +#
>
> No check for NEEDS_LIBINTL? [..]
>
>  +# Define NEEDS_LIBINTL if you haven't set NO_GETTEXT and your system
>  +# needs to be explicitly linked to -lintl. It's defined automatically
>  +# on platforms where we don't expect glibc (Linux, Hurd,
>  +# GNU/kFreeBSD), which includes libintl.
>  +AC_CHECK_LIB([c], [gettext],
>  +[NEEDS_LIBINTL=],
>  +[NEEDS_LIBINTL=YesPlease])
>  +AC_SUBST(NEEDS_LIBINTL)
>  +test -n "$NEEDS_LIBINTL" && LIBS="$LIBS -lintl"
>
> Or something like that (following examples for NEEDS_SOCKET and
> NEEDS_RESOLV in configure.ac).

(I added a NEEDS_RESOLV check in another patch that's exactly like
this, thanks).

> No check that gettext is properly set up with AM_GNU_GETTEXT
> (protected with m4_ifdef)?

Since it was noted before and Ben Walton and David M. Syzdek were CC'd
I thought I'd wait for what they have to say about it:

    http://www.spinics.net/lists/git/msg132431.html

I'm not familiar enough with autoconf to know which'd be better, and I
don't see how to use the AM_GNU_GETTEXT() macro to set and substitute
the NO_GETTEXT variable like the AC_CHECK_HEADER check does now.

Help from someone familiar with autoconf's m4 library would be
appreciated.

>> diff --git a/t/test-lib.sh b/t/test-lib.sh
>> index 454880a..ae63316 100644
>> --- a/t/test-lib.sh
>> +++ b/t/test-lib.sh
>> @@ -37,6 +37,7 @@ ORIGINAL_TERM=$TERM
>>  # For repeatability, reset the environment to known value.
>>  LANG=C
>>  LC_ALL=C
>> +LANGUAGE=C
>>  PAGER=cat
>>  TZ=UTC
>>  TERM=dumb
>
> This ensures that testsuite is run without translation.  It is
> required because tests often include checking output of git commands
> against expected output.

Right, but I found that it didn't work as intended unless LANGUAGE=
was also set, so I added it.

> But perhaps, in later commit, we should mark those test that check
> porcelain output format with NO_GETTEXT or LANG_C, and add --gettext
> or --intl or --localized to run (parts of) testsuite with localized
> strings, checking if localization didn't broke some scripted command
> (somewhere command parses output of other git command).
>
> What do you think?

Yeah, optionally overriding the test suite to not set the LC variables
to test whether something breaks with localized output would be useful
later. It could be done just as:

    if test -z "$GIT_TEST_SET_C_LOCALE"
    then
         LC_ALL=C
         ...
    fi