Git build on antique PowerMac

Git build on antique PowerMac

[Jeffrey Walton]

Hi Everyone,

I have a PowerMac I use for testing. It provides several testing
differentiators, like OS X 10.5, Bash 3.2, GCC 4.0.1, Apple cc-tools
linker, and big-endian PowerPC. (I think Gentoo provides a Linux image
for the hardware, but I don't use it).

The Git libraries and programs build fine out of the box. They also
seem to work as expected once installed.

The pain point is the self self tests, which I have never been able to
build (or execute). I'd like to close this gap.

make -C templates  SHELL_PATH='/bin/sh' PERL_PATH='/usr/bin/perl'
: no custom templates yet
make -C t/ all
rm -f -r 'test-results'
readline() on unopened filehandle       test_must_fail
run_sub_test_lib_test  at check-non-portable-shell.pl line 34.
Modification of a read-only value attempted at
check-non-portable-shell.pl line 34.
make[1]: *** [test-lint-shell-syntax] Error 255

Does anyone want to take a shot at this antique system? I can provide
SSH access with root.

Jeff

Re: Git build on antique PowerMac

[SZEDER Gábor]

On Sun, May 05, 2019 at 03:42:45PM -0400, Jeffrey Walton wrote:
> I have a PowerMac I use for testing. It provides several testing
> differentiators, like OS X 10.5, Bash 3.2, GCC 4.0.1, Apple cc-tools
> linker, and big-endian PowerPC. (I think Gentoo provides a Linux image
> for the hardware, but I don't use it).
> 
> The Git libraries and programs build fine out of the box. They also
> seem to work as expected once installed.
> 
> The pain point is the self self tests, which I have never been able to
> build (or execute). I'd like to close this gap.
> 
> make -C templates  SHELL_PATH='/bin/sh' PERL_PATH='/usr/bin/perl'
> : no custom templates yet
> make -C t/ all
> rm -f -r 'test-results'
> readline() on unopened filehandle       test_must_fail
> run_sub_test_lib_test  at check-non-portable-shell.pl line 34.
> Modification of a read-only value attempted at
> check-non-portable-shell.pl line 34.
> make[1]: *** [test-lint-shell-syntax] Error 255

You could try:

  make -C t all TEST_LINT=

That error comes from one of the "lint" targets that are supposed to
catch some common errors that we repeatedly made in our tests.  This
linter is enabled by default, but this command disables it.

Yeah, it's not the proper solution, but I would think that you don't
need them on that antique PowerMac anyway.

Re: Git build on antique PowerMac

[Eric Sunshine]

On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton <noloader@gmail.com> wrote:
> I have a PowerMac I use for testing. [...]
> make -C t/ all
> readline() on unopened filehandle       test_must_fail
> run_sub_test_lib_test  at check-non-portable-shell.pl line 34.
> Modification of a read-only value attempted at
> check-non-portable-shell.pl line 34.
> make[1]: *** [test-lint-shell-syntax] Error 255
>
> Does anyone want to take a shot at this antique system? I can provide
> SSH access with root.

As the author of the code triggering that problem, I can take a look
at it. (You already have my public SSH key on your Solaris box.)

Given [1], I can see (I guess) why it's complaining about modification
of a read-only variable. However, the unopened filehandle complaint
looks odd.

[1]: https://www.perlmonks.org/?node_id=570712#default_unlocalized

Re: Git build on antique PowerMac

[Ævar Arnfjörð Bjarmason]

On Sun, May 05 2019, Eric Sunshine wrote:

> On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton <noloader@gmail.com> wrote:
>> I have a PowerMac I use for testing. [...]
>> make -C t/ all
>> readline() on unopened filehandle       test_must_fail
>> run_sub_test_lib_test  at check-non-portable-shell.pl line 34.
>> Modification of a read-only value attempted at
>> check-non-portable-shell.pl line 34.
>> make[1]: *** [test-lint-shell-syntax] Error 255
>>
>> Does anyone want to take a shot at this antique system? I can provide
>> SSH access with root.
>
> As the author of the code triggering that problem, I can take a look
> at it. (You already have my public SSH key on your Solaris box.)
>
> Given [1], I can see (I guess) why it's complaining about modification
> of a read-only variable. However, the unopened filehandle complaint
> looks odd.
>
> [1]: https://www.perlmonks.org/?node_id=570712#default_unlocalized

It's because "while (<>) { readline }" didn't use to be supported until
5.10, and he's on 5.8.

    $ seq 1 4 | ./miniperl -we 'print $]; while (<>) { chomp; print "$.: $_ -> " . readline }; print "\n"'
    readline() on unopened filehandle 1: 1 ->  at -e line 1.
    Use of uninitialized value in concatenation (.) or string at -e line 1.
    readline() on unopened filehandle 2: 2 ->  at -e line 1.
    readline() on unopened filehandle 3: 3 ->  at -e line 1.
    readline() on unopened filehandle 4: 4 ->  at -e line 1.
    5.009004
    $ seq 1 4 | perl -we 'print $]; while (<>) { chomp; print "$.: $_ -> " . readline }; print "\n"'
    5.0280011: 1 -> 2
    3: 3 -> 4

Re: Git build on antique PowerMac

[Eric Sunshine]

On Wed, May 8, 2019 at 6:28 PM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Sun, May 05 2019, Eric Sunshine wrote:
> > On Sun, May 5, 2019 at 3:43 PM Jeffrey Walton <noloader@gmail.com> wrote:
> >> readline() on unopened filehandle       test_must_fail
> >
> > Given [1], I can see (I guess) why it's complaining about modification
> > of a read-only variable. However, the unopened filehandle complaint
> > looks odd.
>
> It's because "while (<>) { readline }" didn't use to be supported until
> 5.10, and he's on 5.8.

Thanks for researching this. Now it makes sense.

[PATCH] check-non-portable-shell: support Perl versions older than 5.10

[Eric Sunshine]

For thoroughness when checking for one-shot environment variable
assignments at shell function call sites, check-non-portable-shell
stitches together incomplete lines (those ending with backslash). This
allows it to correctly flag such undesirable usage even when the
variable assignment and function call are split across lines, for
example:

    FOO=bar \
    func

where 'func' is a shell function.

The stitching is accomplished like this:

    while (<>) {
        chomp;
        # stitch together incomplete lines (those ending with "\")
        while (s/\\$//) {
            $_ .= readline;
            chomp;
        }
        # detect unportable/undesirable shell constructs
        ...
    }

Although this implementation is well supported in reasonably modern Perl
versions (5.10 and later), it fails in a couple ways with older versions
(such as Perl 5.8 shipped with ancient Mac OS 10.5).

In particular, in older Perl versions, 'readline' is not connected to
the file handle associated with the "magic" while (<>) {...} construct,
so 'readline' throws a "readline() on unopened filehandle" error.
Furthermore, $_ assigned by the outer while-loop is read-only, so the
attempt to modify it via "$_ .= readline" in the inner while-loop fails
with a "Modification of a read-only value" error.

Avoid both problems by collecting the stitched-together line in a
variable other than $_ and dropping the inner loop entirely.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/check-non-portable-shell.pl | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 166d64d4a2..60e607ba42 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -27,14 +27,14 @@ sub err {
 	close $f;
 }
 
+my $line = '';
 while (<>) {
 	chomp;
+	$line .= $_;
 	# stitch together incomplete lines (those ending with "\")
-	while (s/\\$//) {
-		$_ .= readline;
-		chomp;
-	}
+	next if $line =~ s/\\$//;
 
+	local $_ = $line;
 	/\bcp\s+-a/ and err 'cp -a is not portable';
 	/\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)';
 	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
@@ -48,6 +48,7 @@ sub err {
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
+	$line = '';
 	# this resets our $. for each file
 	close ARGV if eof;
 }
-- 
2.21.0.1084.g81c186ecd2

Re: [PATCH] check-non-portable-shell: support Perl versions older than 5.10

[Ævar Arnfjörð Bjarmason]

On Thu, May 09 2019, Eric Sunshine wrote:

> For thoroughness when checking for one-shot environment variable
> assignments at shell function call sites, check-non-portable-shell
> stitches together incomplete lines (those ending with backslash). This
> allows it to correctly flag such undesirable usage even when the
> variable assignment and function call are split across lines, for
> example:
>
>     FOO=bar \
>     func
>
> where 'func' is a shell function.
>
> The stitching is accomplished like this:
>
>     while (<>) {
>         chomp;
>         # stitch together incomplete lines (those ending with "\")
>         while (s/\\$//) {
>             $_ .= readline;
>             chomp;
>         }
>         # detect unportable/undesirable shell constructs
>         ...
>     }
>
> Although this implementation is well supported in reasonably modern Perl
> versions (5.10 and later), it fails in a couple ways with older versions
> (such as Perl 5.8 shipped with ancient Mac OS 10.5).
>
> In particular, in older Perl versions, 'readline' is not connected to
> the file handle associated with the "magic" while (<>) {...} construct,
> so 'readline' throws a "readline() on unopened filehandle" error.
> Furthermore, $_ assigned by the outer while-loop is read-only, so the
> attempt to modify it via "$_ .= readline" in the inner while-loop fails
> with a "Modification of a read-only value" error.
>
> Avoid both problems by collecting the stitched-together line in a
> variable other than $_ and dropping the inner loop entirely.
>
> Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
> ---
>  t/check-non-portable-shell.pl | 9 +++++----
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
> index 166d64d4a2..60e607ba42 100755
> --- a/t/check-non-portable-shell.pl
> +++ b/t/check-non-portable-shell.pl
> @@ -27,14 +27,14 @@ sub err {
>  	close $f;
>  }
>
> +my $line = '';
>  while (<>) {
>  	chomp;
> +	$line .= $_;
>  	# stitch together incomplete lines (those ending with "\")
> -	while (s/\\$//) {
> -		$_ .= readline;
> -		chomp;
> -	}
> +	next if $line =~ s/\\$//;
>
> +	local $_ = $line;
>  	/\bcp\s+-a/ and err 'cp -a is not portable';
>  	/\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)';
>  	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
> @@ -48,6 +48,7 @@ sub err {
>  	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
>  	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
>  		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
> +	$line = '';
>  	# this resets our $. for each file
>  	close ARGV if eof;
>  }

This fix is fine, but just for the record: There's no problem with
assigning to $_, it just throws an error about $_ *because* of the
readline() issue, i.e. it'll fail, clobber $_ to a read-only value, and
off we go.

So just assigning to $_ is fine, and you don't need to localize it.

Anyway, I tested this on 5.8, it works, then looked at the output and
wondered if I could improve it, came up with this:

    diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
    index 60e607ba42..d5fd0a3050 100755
    --- a/t/check-non-portable-shell.pl
    +++ b/t/check-non-portable-shell.pl
    @@ -8,13 +8,26 @@

     my $exit_code=0;
     my %func;
    +my $start_nr = 0;
    +my $line = '';

     sub err {
     	my $msg = shift;
    -	s/^\s+//;
    -	s/\s+$//;
    -	s/\s+/ /g;
    -	print "$ARGV:$.: error: $msg: $_\n";
    +	if (/\n/) {
    +		$. = $start_nr;
    +		my ($ws) = $_ =~ /^(\s+)/;
    +		for (split /^/) {
    +			s/^\Q$ws\E//;
    +			print "$ARGV:$.: error: $msg: $_";
    +			$.++;
    +		}
    +		print "\n";
    +	} else {
    +		s/^\s+//;
    +		s/\s+$//;
    +		s/\s+/ /g;
    +		print "$ARGV:$.: error: $msg: $_\n";
    +	}
     	$exit_code = 1;
     }

    @@ -27,14 +40,16 @@ sub err {
     	close $f;
     }

    -my $line = '';
     while (<>) {
     	chomp;
    -	$line .= $_;
     	# stitch together incomplete lines (those ending with "\")
    -	next if $line =~ s/\\$//;
    -
    -	local $_ = $line;
    +	if (s/\\$//) {
    +		$start_nr ||= $.;
    +		$line .= "$_\n";
    +		next;
    +	} else {
    +		$_ = $line . $_;
    +	}
     	/\bcp\s+-a/ and err 'cp -a is not portable';
     	/\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)';
     	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
    @@ -48,7 +63,11 @@ sub err {
     	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
     	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
     		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
    +
    +	# No longer spanning lines
    +	$start_nr = 0;
     	$line = '';
    +
     	# this resets our $. for each file
     	close ARGV if eof;
     }
    diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
    index c03054c538..b4af7032ad 100755
    --- a/t/t0000-basic.sh
    +++ b/t/t0000-basic.sh
    @@ -156,8 +156,11 @@ test_expect_success 'pretend we have a fully passing test suite' "
     "

     test_expect_success 'pretend we have a partially passing test suite' "
    -	test_must_fail run_sub_test_lib_test \
    -		partial-pass '2/3 tests passing' <<-\\EOF &&
    +	test_must_fail penis run_sub_test_lib_test \
    +		partial-pass '2/3 tests passing' <<-\\EOF \
    +		partial-pass '2/3 tests passing' <<-\\EOF \
    +		partial-pass '2/3 tests passing' <<-\\EOF \
    +		cp -a hi there &&
     	test_expect_success 'passing test #1' 'true'
     	test_expect_success 'failing test #2' 'false'
     	test_expect_success 'passing test #3' 'true'
    diff --git a/t/t0001-init.sh b/t/t0001-init.sh
    index 1f462204ea..a25ac208e5 100755
    --- a/t/t0001-init.sh
    +++ b/t/t0001-init.sh
    @@ -122,6 +122,7 @@ test_expect_success 'plain bare with GIT_WORK_TREE' '

     test_expect_success 'GIT_DIR bare' '
     	mkdir git-dir-bare.git &&
    +	cp -a foo bar &&
     	GIT_DIR=git-dir-bare.git git init &&
     	check_config git-dir-bare.git true unset
     '

I.e. now for these multi-line issues we'll print the whole offending
multi-line invocation

    $ ~/g/perl/miniperl -I ~/g/perl/lib check-non-portable-shell.pl t[0-9]*.sh
    t0000-basic.sh:159: error: cp -a is not portable: test_must_fail penis run_sub_test_lib_test
    t0000-basic.sh:160: error: cp -a is not portable:       partial-pass '2/3 tests passing' <<-\\EOF
    t0000-basic.sh:161: error: cp -a is not portable:       partial-pass '2/3 tests passing' <<-\\EOF
    t0000-basic.sh:162: error: cp -a is not portable:       partial-pass '2/3 tests passing' <<-\\EOF
    t0000-basic.sh:163: error: cp -a is not portable:       cp -a hi there &&
    t0001-init.sh:125: error: cp -a is not portable: cp -a foo bar &&

I figured it was better than the current output just squashing such a
long line together, i.e. it'll print this now (before/after this patch):

    $ ~/g/perl/miniperl -I ~/g/perl/lib check-non-portable-shell.pl t[0-9]*.sh
    t0000-basic.sh:163: error: cp -a is not portable: test_must_fail penis run_sub_test_lib_test partial-pass '2/3 tests passing' <<-\\EOF partial-pass '2/3 tests passing' <<-\\EOF partial-pass '2/3 tests passing' <<-\\EOF cp -a hi there &&
    t0001-init.sh:125: error: cp -a is not portable: cp -a foo bar &&

There's ways to make that WIP patch shorter etc. I wasn't trying to golf
it, also I think we can get rid of that s/\s+$// if we split up lines
like this.

Re: [PATCH] check-non-portable-shell: support Perl versions older than 5.10

[Eric Sunshine]

On Thu, May 9, 2019 at 8:33 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> On Thu, May 09 2019, Eric Sunshine wrote:
> > Although this implementation is well supported in reasonably modern Perl
> > versions (5.10 and later), it fails in a couple ways with older versions
> > (such as Perl 5.8 shipped with ancient Mac OS 10.5).
> > [...]
> > Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
>
> This fix is fine, but just for the record: There's no problem with
> assigning to $_, it just throws an error about $_ *because* of the
> readline() issue, i.e. it'll fail, clobber $_ to a read-only value, and
> off we go.
> So just assigning to $_ is fine, and you don't need to localize it.

Thanks for the clarification. I'll drop the misleading discussion of
$_ from the commit message and eliminate $_ localization from the
patch proper.

> Anyway, I tested this on 5.8, it works, then looked at the output and
> wondered if I could improve it, came up with this:
> [...large WIP patch snipped...]
> I figured it was better than the current output just squashing such a
> long line together [...]

That could be done, though it is outside the scope of this
portability-fix patch.

Also, the existing output with its sufficiently clear problem
explanations is probably "good enough" to clue in the developer of new
test code as to what needs fixing, so I'm also not sure that it's
worth that much effort to prettify what should be a relatively rare
warning message from code which has not yet made it into the project.
I wouldn't be opposed to someone working on it, but I don't plan on
tackling it myself.

[PATCH v2] check-non-portable-shell: support Perl versions older than 5.10

[Eric Sunshine]

For thoroughness when checking for one-shot environment variable
assignments at shell function call sites, check-non-portable-shell
stitches together incomplete lines (those ending with backslash). This
allows it to correctly flag such undesirable usage even when the
variable assignment and function call are split across lines, for
example:

    FOO=bar \
    func

where 'func' is a shell function.

The stitching is accomplished like this:

    while (<>) {
        chomp;
        # stitch together incomplete lines (those ending with "\")
        while (s/\\$//) {
            $_ .= readline;
            chomp;
        }
        # detect unportable/undesirable shell constructs
        ...
    }

Although this implementation is well supported in reasonably modern Perl
versions (5.10 and later), it fails with older versions (such as Perl
5.8 shipped with ancient Mac OS 10.5). In particular, in older Perl
versions, 'readline' is not connected to the file handle associated with
the "magic" while (<>) {...} construct, so 'readline' throws a
"readline() on unopened filehandle" error. Work around this problem by
dropping readline() and instead incorporating the stitching of
incomplete lines directly into the existing while (<>) {...} loop.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---

This is a re-roll of [1]. The only change since v1 is to drop
localization of $_ as suggested by Ævar in [2], and rewriting the commit
message to avoid misleadingly talking about $_ as being problematic.

[1]: http://public-inbox.org/git/20190509102037.27044-1-sunshine@sunshineco.com/
[2]: http://public-inbox.org/git/87ftpnhknn.fsf@evledraar.gmail.com/

 t/check-non-portable-shell.pl | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 166d64d4a2..38bfeebd88 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -27,14 +27,14 @@ sub err {
 	close $f;
 }
 
+my $line = '';
 while (<>) {
 	chomp;
+	$line .= $_;
 	# stitch together incomplete lines (those ending with "\")
-	while (s/\\$//) {
-		$_ .= readline;
-		chomp;
-	}
+	next if $line =~ s/\\$//;
 
+	$_ = $line;
 	/\bcp\s+-a/ and err 'cp -a is not portable';
 	/\bsed\s+-[^efn]\s+/ and err 'sed option not portable (use only -n, -e, -f)';
 	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
@@ -48,6 +48,7 @@ sub err {
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
+	$line = '';
 	# this resets our $. for each file
 	close ARGV if eof;
 }
-- 
2.21.0.1090.g8b4b5564af