[PATCH] tests: fix and add lint for non-portable head -c N

[PATCH] tests: fix and add lint for non-portable head -c N

[Ævar Arnfjörð Bjarmason]

The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
such invocations to use the test_copy_bytes wrapper added in
48860819e8 ("t9300: factor out portable "head -c" replacement",
2016-06-30).

This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
mmap reads", 2018-06-14), which has been breaking
t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
already have a similar workaround after their upgrade to 2.18.0[2].

I have not tested this on IRIX, but according to 4de0bbd898 ("t9300:
use perl "head -c" clone in place of "dd bs=1 count=16000" kluge",
2010-12-13) this invocation would have broken things there too.

Also, change a valgrind-specific codepath in test-lib.sh to use this
wrapper. Given where valgrind runs I don't think this would ever
become a portability issue in practice, but it's easier to just use
the wrapper than introduce some exception for the "make test-lint"
check being added here.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
2. https://github.com/openbsd/ports/commit/08d5d82eaefe5cf2f125ecc0c6a57df9cf91350c#diff-f7d3c4fabeed1691620d608f1534f5e5

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---

Junio: Even though this isn't a 2.19.0-rc0 regression I think it makes
sense for 2.19.0. The fix is trivial, and it'll unbreak (at least some
of) the tests on stock git on OpenBSD.

OpenBSD guys: If you CC the git mailing list when you find you need to
apply patches like these, we're happy to fix this more pro-actively. I
just happened to be testing the upcoming 2.19.0 on OpenBSD and spotted
this.

 t/check-non-portable-shell.pl | 1 +
 t/t5310-pack-bitmaps.sh       | 2 +-
 t/test-lib.sh                 | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index d5823f71d8..94a7e6165e 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -35,6 +35,7 @@ sub err {
 		chomp;
 	}
 
+	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/\bsed\s+-i/ and err 'sed -i is not portable';
 	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
 	/^\s*declare\s+/ and err 'arrays/declare not portable';
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index 557bd0d0c0..7bff7923f2 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
 	git rev-list --use-bitmap-index --count --all >expect &&
 	bitmap=$(ls .git/objects/pack/*.bitmap) &&
 	test_when_finished "rm -f $bitmap" &&
-	head -c 512 <$bitmap >$bitmap.tmp &&
+	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
 	mv -f $bitmap.tmp $bitmap &&
 	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
 	test_cmp expect actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8bb0f4348e..44288cbb59 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -867,7 +867,7 @@ then
 		# handle only executables, unless they are shell libraries that
 		# need to be in the exec-path.
 		test -x "$1" ||
-		test "# " = "$(head -c 2 <"$1")" ||
+		test "# " = "$(test_copy_bytes 2 <"$1")" ||
 		return;
 
 		base=$(basename "$1")
@@ -882,7 +882,7 @@ then
 		# do not override scripts
 		if test -x "$symlink_target" &&
 		    test ! -d "$symlink_target" &&
-		    test "#!" != "$(head -c 2 < "$symlink_target")"
+		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
 		then
 			symlink_target=../valgrind.sh
 		fi
-- 
2.18.0.865.gffc8e1a3cd6

Test failures on OpenBSD

[Ævar Arnfjörð Bjarmason]

This is on OpenBSD 6.2 amd64 with my "tests: fix and add lint for
non-portable head -c N" patch (which fixes one failure).

    $ for t in t1305-config-include.sh t1308-config-set.sh t5004-archive-corner-cases.sh t5552-skipping-fetch-negotiator.sh; do ./$t --no-color -v -x 2>&1 | grep -B10 "^not ok"; done
    
    + cd bar
    + echo [includeIf "gitdir:bar/"]path=bar7
    + >> .git/config 
    + echo [test]seven=7
    + > .git/bar7 
    + echo 7
    + > expect 
    + git config test.seven
    + > actual 
    error: last command exited with $?=1
    not ok 27 - conditional include, gitdir matching symlink
    --
    + cd bar
    + echo [includeIf "gitdir/i:BAR/"]path=bar8
    + >> .git/config 
    + echo [test]eight=8
    + > .git/bar8 
    + echo 8
    + > expect 
    + git config test.eight
    + > actual 
    error: last command exited with $?=1
    not ok 28 - conditional include, gitdir matching symlink, icase
    	test_cmp expect actual
    
    + echo Error (-1) reading configuration file a-directory.
    + > expect 
    + mkdir a-directory
    + test_expect_code 2 test-tool config configset_get_value foo.bar a-directory
    + 2> output 
    Value not found for "foo.bar"
    test_expect_code: command exited with 1, we wanted 2 test-tool config configset_get_value foo.bar a-directory
    error: last command exited with $?=1
    not ok 23 - proper error on directory "files"
    + make_dir extract
    + tar xf subtree-all.tar -C extract
    tar: Cannot identify format. Searching...
    tar: End of archive volume 1 reached
    tar: Sorry, unable to determine archive format.
    error: last command exited with $?=1
    + rm -rf extract
    + exit 1
    + eval_ret=1
    + :
    not ok 9 - archive empty subtree with no pathspec
    --
    + make_dir extract
    + tar xf subtree-path.tar -C extract
    tar: Cannot identify format. Searching...
    tar: End of archive volume 1 reached
    tar: Sorry, unable to determine archive format.
    error: last command exited with $?=1
    + rm -rf extract
    + exit 1
    + eval_ret=1
    + :
    not ok 10 - archive empty subtree by direct pathspec
    'git <command> [<revision>...] -- [<file>...]'
    fatal: ambiguous argument 'c7': unknown revision or path not in the working tree.
    Use '--' to separate paths from revisions, like this:
    'git <command> [<revision>...] -- [<file>...]'
    No have c7 (c7)
    error: last command exited with $?=1
    + test_unconfig -C client fetch.negotiationalgorithm
    + exit 1
    + eval_ret=1
    + :
    not ok 1 - commits with no parents are sent regardless of skip distance
    --
     Author: A U Thor <author@example.com>
     1 file changed, 1 insertion(+)
     create mode 100644 to_fetch.t
    + git init client
    Initialized empty Git repository in /home/avar/g/git/t/trash directory.t5552-skipping-fetch-negotiator/client/.git/
    + seq 11
    ./t5552-skipping-fetch-negotiator.sh: seq: not found
    + git -C client checkout c5
    error: pathspec 'c5' did not match any file(s) known to git
    error: last command exited with $?=1
    not ok 3 - when two skips collide, favor the larger one
    --
    + git init client
    Initialized empty Git repository in /home/avar/g/git/t/trash directory.t5552-skipping-fetch-negotiator/client/.git/
    + seq 8
    ./t5552-skipping-fetch-negotiator.sh: seq: not found
    + seq 19
    ./t5552-skipping-fetch-negotiator.sh: seq: not found
    + pwd
    + git -C server fetch --no-tags /home/avar/g/git/t/trash directory.t5552-skipping-fetch-negotiator/client b1:refs/heads/b1
    fatal: Couldn't find remote ref b1
    error: last command exited with $?=128
    not ok 6 - do not send "have" with ancestors of commits that server ACKed

Full output at https://gitlab.com/snippets/1747801

Some of this, like the t5552-skipping-fetch-negotiator.sh failures are
new in 2.19.0 (those go away with s/seq/test_seq). But some are
older. E.g. the archive corner cases failure is becuse that test wants
unzip & GNU tar, which OpenBSD upstream has patched already:
https://github.com/openbsd/ports/blob/master/devel/git/patches/patch-t_test-lib_sh

Re: Test failures on OpenBSD

[Junio C Hamano]

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>     + echo Error (-1) reading configuration file a-directory.
>     + > expect 
>     + mkdir a-directory
>     + test_expect_code 2 test-tool config configset_get_value foo.bar a-directory
>     + 2> output 
>     Value not found for "foo.bar"
>     test_expect_code: command exited with 1, we wanted 2 test-tool config configset_get_value foo.bar a-directory
>     error: last command exited with $?=1
>     not ok 23 - proper error on directory "files"

A blind guess.  is this "BSD allows opening a FILE* on a directory
and reading bytes from it" aka FREAD_READS_DIRECTORIES?

I wonder why FreeBSD sets it but not OpenBSD in config.mak.uname.

>     + make_dir extract
>     + tar xf subtree-all.tar -C extract
>     tar: Cannot identify format. Searching...
>     tar: End of archive volume 1 reached
>     tar: Sorry, unable to determine archive format.
>     error: last command exited with $?=1

Perhaps modern tar elements like extended headers are not supported
on the platform-supplied "tar"?  "make TAR=gnutar"?

>     + seq 11
>     ./t5552-skipping-fetch-negotiator.sh: seq: not found

This must use test_seq.

 t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh
index 5ad5bece55..30857b84a8 100755
--- a/t/t5552-skipping-fetch-negotiator.sh
+++ b/t/t5552-skipping-fetch-negotiator.sh
@@ -46,7 +46,7 @@ test_expect_success 'commits with no parents are sent regardless of skip distanc
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 7)
+	for i in $(test_seq 7)
 	do
 		test_commit -C client c$i
 	done &&
@@ -89,7 +89,7 @@ test_expect_success 'when two skips collide, favor the larger one' '
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 11)
+	for i in $(test_seq 11)
 	do
 		test_commit -C client c$i
 	done &&
@@ -168,14 +168,14 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 8)
+	for i in $(test_seq 8)
 	do
 		git -C client checkout --orphan b$i &&
 		test_commit -C client b$i.c0
 	done &&
-	for j in $(seq 19)
+	for j in $(test_seq 19)
 	do
-		for i in $(seq 8)
+		for i in $(test_seq 8)
 		do
 			git -C client checkout b$i &&
 			test_commit -C client b$i.c$j
@@ -205,7 +205,7 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 
 	# fetch-pack should thus not send any more commits in the b1 branch, but
 	# should still send the others (in this test, just check b2).
-	for i in $(seq 0 8)
+	for i in $(test_seq 0 8)
 	do
 		have_not_sent b1.c$i
 	done &&

[PATCH v2 1/2] tests: fix and add lint for non-portable head -c N

[Ævar Arnfjörð Bjarmason]

The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
such invocations to use the test_copy_bytes wrapper added in
48860819e8 ("t9300: factor out portable "head -c" replacement",
2016-06-30).

This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
mmap reads", 2018-06-14), which has been breaking
t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
already have a similar workaround after their upgrade to 2.18.0[2].

I have not tested this on IRIX, but according to 4de0bbd898 ("t9300:
use perl "head -c" clone in place of "dd bs=1 count=16000" kluge",
2010-12-13) this invocation would have broken things there too.

Also, change a valgrind-specific codepath in test-lib.sh to use this
wrapper. Given where valgrind runs I don't think this would ever
become a portability issue in practice, but it's easier to just use
the wrapper than introduce some exception for the "make test-lint"
check being added here.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
2. https://github.com/openbsd/ports/commit/08d5d82eaefe5cf2f125ecc0c6a57df9cf91350c#diff-f7d3c4fabeed1691620d608f1534f5e5

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl | 1 +
 t/t5310-pack-bitmaps.sh       | 2 +-
 t/test-lib.sh                 | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index d5823f71d8..c8f10d40a1 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -41,6 +41,7 @@ sub err {
 	/^\s*[^#]\s*which\s/ and err 'which is not portable (use type)';
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
+	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index 557bd0d0c0..7bff7923f2 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
 	git rev-list --use-bitmap-index --count --all >expect &&
 	bitmap=$(ls .git/objects/pack/*.bitmap) &&
 	test_when_finished "rm -f $bitmap" &&
-	head -c 512 <$bitmap >$bitmap.tmp &&
+	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
 	mv -f $bitmap.tmp $bitmap &&
 	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
 	test_cmp expect actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8bb0f4348e..44288cbb59 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -867,7 +867,7 @@ then
 		# handle only executables, unless they are shell libraries that
 		# need to be in the exec-path.
 		test -x "$1" ||
-		test "# " = "$(head -c 2 <"$1")" ||
+		test "# " = "$(test_copy_bytes 2 <"$1")" ||
 		return;
 
 		base=$(basename "$1")
@@ -882,7 +882,7 @@ then
 		# do not override scripts
 		if test -x "$symlink_target" &&
 		    test ! -d "$symlink_target" &&
-		    test "#!" != "$(head -c 2 < "$symlink_target")"
+		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
 		then
 			symlink_target=../valgrind.sh
 		fi
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v2 1/2] tests: fix and add lint for non-portable head -c N

[Jeff King]

On Thu, Aug 23, 2018 at 03:25:01PM +0000, Ævar Arnfjörð Bjarmason wrote:

> The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
> such invocations to use the test_copy_bytes wrapper added in
> 48860819e8 ("t9300: factor out portable "head -c" replacement",
> 2016-06-30).
> 
> This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
> mmap reads", 2018-06-14), which has been breaking
> t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
> already have a similar workaround after their upgrade to 2.18.0[2].

Heh, I even considered using this when writing that test. But the reason
I introduced test_copy_bytes is not because the target platform did not
have "head -c" at all, but because some tests need very specific
buffering guarantees when reading from a shared pipe.

That said, if OpenBSD's "head" doesn't have "-c" at all, I'm fine with
this as a fix (and it sounds like we know that IRIX lacks it, too).

> Also, change a valgrind-specific codepath in test-lib.sh to use this
> wrapper. Given where valgrind runs I don't think this would ever
> become a portability issue in practice, but it's easier to just use
> the wrapper than introduce some exception for the "make test-lint"
> check being added here.

When working on 9d2e330b17, I recall finding these other "head -c"
invocations in the test suite when I did 9d2e330b17 and took them as
evidence that it was OK to use in vanilla cases. So even if these sites
don't affect any platforms in practice, I think it's worth it to ban
"head -c" completely.

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/check-non-portable-shell.pl | 1 +
>  t/t5310-pack-bitmaps.sh       | 2 +-
>  t/test-lib.sh                 | 4 ++--
>  3 files changed, 4 insertions(+), 3 deletions(-)

Patch itself looks good to me. Thanks.

-Peff

[PATCH v2 2/2] tests: fix and add lint for non-portable seq

[Ævar Arnfjörð Bjarmason]

GNU seq is not a POSIX command, and doesn't exist on
e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
Introduce test_seq", 2012-08-04), but use of it keeps coming back,
e.g. in the recently added "fetch negotiator" tests being added here.

So let's also add a check to "make test-lint". The regex is aiming to
capture the likes of $(seq ..) and "seq" as a stand-alone command,
without capturing some existing cases where we e.g. have files called
"seq".

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---

Now with a fix & check in v2 for the seq issue mentioned in
https://public-inbox.org/git/87a7pdfltp.fsf@evledraar.gmail.com/

 t/check-non-portable-shell.pl        |  1 +
 t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
 t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index c8f10d40a1..75f38298d7 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -42,6 +42,7 @@ sub err {
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
 	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
+	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh
index 5ad5bece55..30857b84a8 100755
--- a/t/t5552-skipping-fetch-negotiator.sh
+++ b/t/t5552-skipping-fetch-negotiator.sh
@@ -46,7 +46,7 @@ test_expect_success 'commits with no parents are sent regardless of skip distanc
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 7)
+	for i in $(test_seq 7)
 	do
 		test_commit -C client c$i
 	done &&
@@ -89,7 +89,7 @@ test_expect_success 'when two skips collide, favor the larger one' '
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 11)
+	for i in $(test_seq 11)
 	do
 		test_commit -C client c$i
 	done &&
@@ -168,14 +168,14 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 8)
+	for i in $(test_seq 8)
 	do
 		git -C client checkout --orphan b$i &&
 		test_commit -C client b$i.c0
 	done &&
-	for j in $(seq 19)
+	for j in $(test_seq 19)
 	do
-		for i in $(seq 8)
+		for i in $(test_seq 8)
 		do
 			git -C client checkout b$i &&
 			test_commit -C client b$i.c$j
@@ -205,7 +205,7 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 
 	# fetch-pack should thus not send any more commits in the b1 branch, but
 	# should still send the others (in this test, just check b2).
-	for i in $(seq 0 8)
+	for i in $(test_seq 0 8)
 	do
 		have_not_sent b1.c$i
 	done &&
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index a73c55a47e..d1ccc22331 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -176,7 +176,7 @@ test_expect_success 'setup repos for change-while-negotiating test' '
 		git clone "http://127.0.0.1:$LIB_HTTPD_PORT/smart/repo" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
@@ -289,7 +289,7 @@ test_expect_success 'setup repos for fetching with ref-in-want tests' '
 		git clone "file://$REPO" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v2 2/2] tests: fix and add lint for non-portable seq

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> GNU seq is not a POSIX command, and doesn't exist on

s/GNU //; the command did not even originate there, but came from V8
and/or Plan9 IIRC.

> e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
> Introduce test_seq", 2012-08-04), but use of it keeps coming back,
> e.g. in the recently added "fetch negotiator" tests being added here.
>
> So let's also add a check to "make test-lint". The regex is aiming to
> capture the likes of $(seq ..) and "seq" as a stand-alone command,
> without capturing some existing cases where we e.g. have files called
> "seq".
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>
> Now with a fix & check in v2 for the seq issue mentioned in
> https://public-inbox.org/git/87a7pdfltp.fsf@evledraar.gmail.com/

Thanks.

>  t/check-non-portable-shell.pl        |  1 +
>  t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
>  t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
>  3 files changed, 9 insertions(+), 8 deletions(-)
>
> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
> index c8f10d40a1..75f38298d7 100755
> --- a/t/check-non-portable-shell.pl
> +++ b/t/check-non-portable-shell.pl
> @@ -42,6 +42,7 @@ sub err {
>  	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
>  	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
>  	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
> +	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';

Looking at $(wc -l) thing a few lines above, I am not sure if this
deviation is a good idea.  Wouldn't /\bseq\s/ be sufficient?

Re: [PATCH v2 2/2] tests: fix and add lint for non-portable seq

[Ævar Arnfjörð Bjarmason]

On Thu, Aug 23 2018, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> GNU seq is not a POSIX command, and doesn't exist on
>
> s/GNU //; the command did not even originate there, but came from V8
> and/or Plan9 IIRC.
>
>> e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
>> Introduce test_seq", 2012-08-04), but use of it keeps coming back,
>> e.g. in the recently added "fetch negotiator" tests being added here.
>>
>> So let's also add a check to "make test-lint". The regex is aiming to
>> capture the likes of $(seq ..) and "seq" as a stand-alone command,
>> without capturing some existing cases where we e.g. have files called
>> "seq".
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>
>> Now with a fix & check in v2 for the seq issue mentioned in
>> https://public-inbox.org/git/87a7pdfltp.fsf@evledraar.gmail.com/
>
> Thanks.
>
>>  t/check-non-portable-shell.pl        |  1 +
>>  t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
>>  t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
>>  3 files changed, 9 insertions(+), 8 deletions(-)
>>
>> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
>> index c8f10d40a1..75f38298d7 100755
>> --- a/t/check-non-portable-shell.pl
>> +++ b/t/check-non-portable-shell.pl
>> @@ -42,6 +42,7 @@ sub err {
>>  	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
>>  	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
>>  	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
>> +	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
>
> Looking at $(wc -l) thing a few lines above, I am not sure if this
> deviation is a good idea.  Wouldn't /\bseq\s/ be sufficient?

As alluded to in the commit message that will find cases like:

    $ git grep -P '\bseq\b' -- t | grep -P -v '(?:\$\(seq|^\s*seq\b)'
    t/perf/p3400-rebase.sh:16:              seq 1000 >>unrelated-file$i &&
    t/perf/p3400-rebase.sh:21:              seq 1000 | tac >>unrelated-file$i &&
    t/t3404-rebase-interactive.sh:1227:             test_seq 5 | sed "s/$double/&&/" >seq &&
    t/t3404-rebase-interactive.sh:1228:             git add seq &&
    t/t3404-rebase-interactive.sh:1230:             git commit -m seq-$double
    t/t3404-rebase-interactive.sh:1232:     git tag seq-onto &&
    t/t3404-rebase-interactive.sh:1234:     git cherry-pick seq-onto &&
    t/t3404-rebase-interactive.sh:1236:     test_must_fail env FAKE_LINES= git rebase -i seq-onto &&
    t/t3404-rebase-interactive.sh:1239:     git diff --exit-code seq-onto &&
    t/t4022-diff-rewrite.sh:69:     test_seq 1 99 >seq &&
    t/t4022-diff-rewrite.sh:70:     printf 100 >>seq &&
    t/t4022-diff-rewrite.sh:71:     git add seq &&
    t/t4022-diff-rewrite.sh:72:     git commit seq -m seq
    t/t4022-diff-rewrite.sh:76:     test_seq 1 5 >seq &&
    t/t4022-diff-rewrite.sh:77:     test_seq 9331 9420 >>seq &&
    t/t4022-diff-rewrite.sh:78:     test_seq 96 100 >>seq
    t/t4022-diff-rewrite.sh:82:     git diff -B seq >res &&
    t/t4022-diff-rewrite.sh:87:     git diff seq >res &&
    t/t4022-diff-rewrite.sh:93:     git diff -B seq >res &&
    t/t4150-am.sh:933:      git format-patch -2 --stdout >seq.patch &&
    t/t4150-am.sh:945:      test_must_fail git am -3 seq.patch &&
    t/t4150-am.sh:955:      test_must_fail git am -3 seq.patch &&
    t/t5520-pull.sh:295:    git checkout -b seq &&
    t/t5520-pull.sh:296:    test_seq 5 >seq.txt &&
    t/t5520-pull.sh:297:    git add seq.txt &&
    t/t5520-pull.sh:299:    git commit -m "Add seq.txt" &&
    t/t5520-pull.sh:300:    echo 6 >>seq.txt &&
    t/t5520-pull.sh:302:    git commit -m "Append to seq.txt" seq.txt &&
    t/t5520-pull.sh:304:    echo conflicting >>seq.txt &&
    t/t5520-pull.sh:306:    git commit -m "Create conflict" seq.txt &&
    t/t5520-pull.sh:307:    test_must_fail git pull --rebase . seq 2>err >out &&

[PATCH v3 0/5] OpenBSD & AIX etc. portability fixes

[Ævar Arnfjörð Bjarmason]

This grew a bit more. I'm going to stop poking at this for now. The
tests are still broken on OpenBSD (3-5 broken) and on AIX something
like 20-30 are broken, but this makes it slightly better.

Ævar Arnfjörð Bjarmason (5):
  tests: fix and add lint for non-portable head -c N
  tests: fix and add lint for non-portable seq
  tests: use shorter here-docs in chainlint.sed for AIX sed
  tests: fix version-specific portability issue in Perl JSON
  tests: fix and add lint for non-portable grep --file

 t/chainlint.sed                      |  8 ++++----
 t/check-non-portable-shell.pl        |  3 +++
 t/t0019/parse_json.perl              |  3 +++
 t/t5310-pack-bitmaps.sh              |  2 +-
 t/t5318-commit-graph.sh              |  2 +-
 t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
 t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
 t/test-lib.sh                        |  4 ++--
 8 files changed, 22 insertions(+), 16 deletions(-)

-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v3 1/5] tests: fix and add lint for non-portable head -c N

[Ævar Arnfjörð Bjarmason]

The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
such invocations to use the test_copy_bytes wrapper added in
48860819e8 ("t9300: factor out portable "head -c" replacement",
2016-06-30).

This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
mmap reads", 2018-06-14), which has been breaking
t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
already have a similar workaround after their upgrade to 2.18.0[2].

I have not tested this on IRIX, but according to 4de0bbd898 ("t9300:
use perl "head -c" clone in place of "dd bs=1 count=16000" kluge",
2010-12-13) this invocation would have broken things there too.

Also, change a valgrind-specific codepath in test-lib.sh to use this
wrapper. Given where valgrind runs I don't think this would ever
become a portability issue in practice, but it's easier to just use
the wrapper than introduce some exception for the "make test-lint"
check being added here.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
2. https://github.com/openbsd/ports/commit/08d5d82eaefe5cf2f125ecc0c6a57df9cf91350c#diff-f7d3c4fabeed1691620d608f1534f5e5

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl | 1 +
 t/t5310-pack-bitmaps.sh       | 2 +-
 t/test-lib.sh                 | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index d5823f71d8..c8f10d40a1 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -41,6 +41,7 @@ sub err {
 	/^\s*[^#]\s*which\s/ and err 'which is not portable (use type)';
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
+	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index 557bd0d0c0..7bff7923f2 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
 	git rev-list --use-bitmap-index --count --all >expect &&
 	bitmap=$(ls .git/objects/pack/*.bitmap) &&
 	test_when_finished "rm -f $bitmap" &&
-	head -c 512 <$bitmap >$bitmap.tmp &&
+	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
 	mv -f $bitmap.tmp $bitmap &&
 	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
 	test_cmp expect actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8bb0f4348e..44288cbb59 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -867,7 +867,7 @@ then
 		# handle only executables, unless they are shell libraries that
 		# need to be in the exec-path.
 		test -x "$1" ||
-		test "# " = "$(head -c 2 <"$1")" ||
+		test "# " = "$(test_copy_bytes 2 <"$1")" ||
 		return;
 
 		base=$(basename "$1")
@@ -882,7 +882,7 @@ then
 		# do not override scripts
 		if test -x "$symlink_target" &&
 		    test ! -d "$symlink_target" &&
-		    test "#!" != "$(head -c 2 < "$symlink_target")"
+		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
 		then
 			symlink_target=../valgrind.sh
 		fi
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v3 2/5] tests: fix and add lint for non-portable seq

[Ævar Arnfjörð Bjarmason]

The seq command is not in POSIX, and doesn't exist on
e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
Introduce test_seq", 2012-08-04), but use of it keeps coming back,
e.g. in the recently added "fetch negotiator" tests being added here.

So let's also add a check to "make test-lint". The regex is aiming to
capture the likes of $(seq ..) and "seq" as a stand-alone command,
without capturing some existing cases where we e.g. have files called
"seq", as \bseq\b would do.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl        |  1 +
 t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
 t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index c8f10d40a1..75f38298d7 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -42,6 +42,7 @@ sub err {
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
 	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
+	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh
index 5ad5bece55..30857b84a8 100755
--- a/t/t5552-skipping-fetch-negotiator.sh
+++ b/t/t5552-skipping-fetch-negotiator.sh
@@ -46,7 +46,7 @@ test_expect_success 'commits with no parents are sent regardless of skip distanc
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 7)
+	for i in $(test_seq 7)
 	do
 		test_commit -C client c$i
 	done &&
@@ -89,7 +89,7 @@ test_expect_success 'when two skips collide, favor the larger one' '
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 11)
+	for i in $(test_seq 11)
 	do
 		test_commit -C client c$i
 	done &&
@@ -168,14 +168,14 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 8)
+	for i in $(test_seq 8)
 	do
 		git -C client checkout --orphan b$i &&
 		test_commit -C client b$i.c0
 	done &&
-	for j in $(seq 19)
+	for j in $(test_seq 19)
 	do
-		for i in $(seq 8)
+		for i in $(test_seq 8)
 		do
 			git -C client checkout b$i &&
 			test_commit -C client b$i.c$j
@@ -205,7 +205,7 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 
 	# fetch-pack should thus not send any more commits in the b1 branch, but
 	# should still send the others (in this test, just check b2).
-	for i in $(seq 0 8)
+	for i in $(test_seq 0 8)
 	do
 		have_not_sent b1.c$i
 	done &&
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index a73c55a47e..d1ccc22331 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -176,7 +176,7 @@ test_expect_success 'setup repos for change-while-negotiating test' '
 		git clone "http://127.0.0.1:$LIB_HTTPD_PORT/smart/repo" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
@@ -289,7 +289,7 @@ test_expect_success 'setup repos for fetching with ref-in-want tests' '
 		git clone "file://$REPO" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v3 3/5] tests: use shorter here-docs in chainlint.sed for AIX sed

[Ævar Arnfjörð Bjarmason]

Improve the portability of chainlint by using shorter here-docs. On
AIX sed will complain about:

    sed: 0602-417 The label :hereslurp is greater than eight
    characters

As noted in [1] there's still a remaining recently introduced
portability issue also introduced in 878f988350 ("t/test-lib: teach
--chain-lint to detect broken &&-chains in subshells", 2018-07-11), so
under AIX the tests must be run with GIT_TEST_CHAIN_LINT=0.

I don't know how to solve the other issue, and this gets us some of
the way to GIT_TEST_CHAIN_LINT=1 working again on AIX.

1. https://public-inbox.org/git/871sapezba.fsf@evledraar.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/chainlint.sed | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/chainlint.sed b/t/chainlint.sed
index 8544df38df..2333705b27 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -97,11 +97,11 @@
 /<<[ 	]*[-\\']*[A-Za-z0-9_]/ {
 	s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
 	s/[ 	]*<<//
-	:hereslurp
+	:hered
 	N
 	/^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
 		s/\n.*$//
-		bhereslurp
+		bhered
 	}
 	s/^<[^>]*>//
 	s/\n.*$//
@@ -283,11 +283,11 @@ bfolded
 :heredoc
 s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
 s/[ 	]*<<//
-:hereslurpsub
+:heredsub
 N
 /^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
 	s/\n.*$//
-	bhereslurpsub
+	bheredsub
 }
 s/^<[^>]*>//
 s/\n.*$//
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v3 3/5] tests: use shorter here-docs in chainlint.sed for AIX sed

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Improve the portability of chainlint by using shorter here-docs. On
> AIX sed will complain about:
>
>     sed: 0602-417 The label :hereslurp is greater than eight
>     characters
>
> As noted in [1] there's still a remaining recently introduced
> portability issue also introduced in 878f988350 ("t/test-lib: teach
> --chain-lint to detect broken &&-chains in subshells", 2018-07-11), so
> under AIX the tests must be run with GIT_TEST_CHAIN_LINT=0.
>
> I don't know how to solve the other issue, and this gets us some of
> the way to GIT_TEST_CHAIN_LINT=1 working again on AIX.
>
> 1. https://public-inbox.org/git/871sapezba.fsf@evledraar.gmail.com/
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---

I'll globally do s/here-doc/label/ while queueing.

POSIX says "The implementation shall support label arguments
recognized as unique up to at least 8 bytes", so replacing these
labels to shorter strings makes perfect sense.

Will queue; thanks.

>  t/chainlint.sed | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/t/chainlint.sed b/t/chainlint.sed
> index 8544df38df..2333705b27 100644
> --- a/t/chainlint.sed
> +++ b/t/chainlint.sed
> @@ -97,11 +97,11 @@
>  /<<[ 	]*[-\\']*[A-Za-z0-9_]/ {
>  	s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
>  	s/[ 	]*<<//
> -	:hereslurp
> +	:hered
>  	N
>  	/^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
>  		s/\n.*$//
> -		bhereslurp
> +		bhered
>  	}
>  	s/^<[^>]*>//
>  	s/\n.*$//
> @@ -283,11 +283,11 @@ bfolded
>  :heredoc
>  s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
>  s/[ 	]*<<//
> -:hereslurpsub
> +:heredsub
>  N
>  /^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
>  	s/\n.*$//
> -	bhereslurpsub
> +	bheredsub
>  }
>  s/^<[^>]*>//
>  s/\n.*$//

Re: [PATCH v3 3/5] tests: use shorter here-docs in chainlint.sed for AIX sed

[Eric Sunshine]

On Thu, Aug 23, 2018 at 4:36 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> As noted in [1] there's still a remaining recently introduced
> portability issue also introduced in 878f988350 ("t/test-lib: teach
> --chain-lint to detect broken &&-chains in subshells", 2018-07-11), so
> under AIX the tests must be run with GIT_TEST_CHAIN_LINT=0.
>
> I don't know how to solve the other issue, and this gets us some of
> the way to GIT_TEST_CHAIN_LINT=1 working again on AIX.

Does unindenting the comment, as I suggested in [1], fix the remaining
problem for you?

[1]: https://public-inbox.org/git/CAPig+cTTbU5HFMKgNyrxTp3+kcK46-Fn=4ZH6zDt1oQChAc3KA@mail.gmail.com/

[PATCH v4 0/6] OpenBSD & AIX etc. portability fixes

[Ævar Arnfjörð Bjarmason]

On Thu, Aug 23 2018, Eric Sunshine wrote:

> On Thu, Aug 23, 2018 at 4:36 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> As noted in [1] there's still a remaining recently introduced
>> portability issue also introduced in 878f988350 ("t/test-lib: teach
>> --chain-lint to detect broken &&-chains in subshells", 2018-07-11), so
>> under AIX the tests must be run with GIT_TEST_CHAIN_LINT=0.
>>
>> I don't know how to solve the other issue, and this gets us some of
>> the way to GIT_TEST_CHAIN_LINT=1 working again on AIX.
>
> Does unindenting the comment, as I suggested in [1], fix the remaining
> problem for you?
>
> [1]: https://public-inbox.org/git/CAPig+cTTbU5HFMKgNyrxTp3+kcK46-Fn=4ZH6zDt1oQChAc3KA@mail.gmail.com/

I didn't notice Eric's E-mail before I sent my v3, so going back and
testing this revealed two things:

 1) Yes, his suggestion works
 2) AIX sed will complain about one thing at a time, so we had a lot
    more of these "labels too long" problems once I got past fixing
    that issue.

So here's a version, which as noted in 4/6 makes GIT_TEST_CHAIN_LINT=1
fully work on AIX again.

As an aside, the reason I have access to AIX is because I requested
access to the GCC compile farm as suggested by someone on-list here
the other day: https://cfarm.tetaneutral.net/

They accepted my account request on the basis that I was going to hack
on git & perl on those boxes, so if anyone else here is interested in
testing stuff for portability...

Ævar Arnfjörð Bjarmason (6):
  tests: fix and add lint for non-portable head -c N
  tests: fix and add lint for non-portable seq
  tests: fix comment syntax in chainlint.sed for AIX sed
  tests: use shorter here-docs in chainlint.sed for AIX sed
  tests: fix version-specific portability issue in Perl JSON
  tests: fix and add lint for non-portable grep --file

 t/chainlint.sed                      | 59 ++++++++++++++--------------
 t/check-non-portable-shell.pl        |  3 ++
 t/t0019/parse_json.perl              |  3 ++
 t/t5310-pack-bitmaps.sh              |  2 +-
 t/t5318-commit-graph.sh              |  2 +-
 t/t5552-skipping-fetch-negotiator.sh | 12 +++---
 t/t5703-upload-pack-ref-in-want.sh   |  4 +-
 t/test-lib.sh                        |  4 +-
 8 files changed, 47 insertions(+), 42 deletions(-)

-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v4 1/6] tests: fix and add lint for non-portable head -c N

[Ævar Arnfjörð Bjarmason]

The "head -c BYTES" option is non-portable (not in POSIX[1]). Change
such invocations to use the test_copy_bytes wrapper added in
48860819e8 ("t9300: factor out portable "head -c" replacement",
2016-06-30).

This fixes a test added in 9d2e330b17 ("ewah_read_mmap: bounds-check
mmap reads", 2018-06-14), which has been breaking
t5310-pack-bitmaps.sh on OpenBSD since 2.18.0. The OpenBSD ports
already have a similar workaround after their upgrade to 2.18.0[2].

I have not tested this on IRIX, but according to 4de0bbd898 ("t9300:
use perl "head -c" clone in place of "dd bs=1 count=16000" kluge",
2010-12-13) this invocation would have broken things there too.

Also, change a valgrind-specific codepath in test-lib.sh to use this
wrapper. Given where valgrind runs I don't think this would ever
become a portability issue in practice, but it's easier to just use
the wrapper than introduce some exception for the "make test-lint"
check being added here.

1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/head.html
2. https://github.com/openbsd/ports/commit/08d5d82eaefe5cf2f125ecc0c6a57df9cf91350c#diff-f7d3c4fabeed1691620d608f1534f5e5

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl | 1 +
 t/t5310-pack-bitmaps.sh       | 2 +-
 t/test-lib.sh                 | 4 ++--
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index d5823f71d8..c8f10d40a1 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -41,6 +41,7 @@ sub err {
 	/^\s*[^#]\s*which\s/ and err 'which is not portable (use type)';
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
+	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index 557bd0d0c0..7bff7923f2 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
 	git rev-list --use-bitmap-index --count --all >expect &&
 	bitmap=$(ls .git/objects/pack/*.bitmap) &&
 	test_when_finished "rm -f $bitmap" &&
-	head -c 512 <$bitmap >$bitmap.tmp &&
+	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
 	mv -f $bitmap.tmp $bitmap &&
 	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
 	test_cmp expect actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 8bb0f4348e..44288cbb59 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -867,7 +867,7 @@ then
 		# handle only executables, unless they are shell libraries that
 		# need to be in the exec-path.
 		test -x "$1" ||
-		test "# " = "$(head -c 2 <"$1")" ||
+		test "# " = "$(test_copy_bytes 2 <"$1")" ||
 		return;
 
 		base=$(basename "$1")
@@ -882,7 +882,7 @@ then
 		# do not override scripts
 		if test -x "$symlink_target" &&
 		    test ! -d "$symlink_target" &&
-		    test "#!" != "$(head -c 2 < "$symlink_target")"
+		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
 		then
 			symlink_target=../valgrind.sh
 		fi
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v4 2/6] tests: fix and add lint for non-portable seq

[Ævar Arnfjörð Bjarmason]

The seq command is not in POSIX, and doesn't exist on
e.g. OpenBSD. We've had the test_seq wrapper since d17cf5f3a3 ("tests:
Introduce test_seq", 2012-08-04), but use of it keeps coming back,
e.g. in the recently added "fetch negotiator" tests being added here.

So let's also add a check to "make test-lint". The regex is aiming to
capture the likes of $(seq ..) and "seq" as a stand-alone command,
without capturing some existing cases where we e.g. have files called
"seq", as \bseq\b would do.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl        |  1 +
 t/t5552-skipping-fetch-negotiator.sh | 12 ++++++------
 t/t5703-upload-pack-ref-in-want.sh   |  4 ++--
 3 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index c8f10d40a1..75f38298d7 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -42,6 +42,7 @@ sub err {
 	/\btest\s+[^=]*==/ and err '"test a == b" is not portable (use =)';
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
 	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
+	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh
index 5ad5bece55..30857b84a8 100755
--- a/t/t5552-skipping-fetch-negotiator.sh
+++ b/t/t5552-skipping-fetch-negotiator.sh
@@ -46,7 +46,7 @@ test_expect_success 'commits with no parents are sent regardless of skip distanc
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 7)
+	for i in $(test_seq 7)
 	do
 		test_commit -C client c$i
 	done &&
@@ -89,7 +89,7 @@ test_expect_success 'when two skips collide, favor the larger one' '
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 11)
+	for i in $(test_seq 11)
 	do
 		test_commit -C client c$i
 	done &&
@@ -168,14 +168,14 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 	test_commit -C server to_fetch &&
 
 	git init client &&
-	for i in $(seq 8)
+	for i in $(test_seq 8)
 	do
 		git -C client checkout --orphan b$i &&
 		test_commit -C client b$i.c0
 	done &&
-	for j in $(seq 19)
+	for j in $(test_seq 19)
 	do
-		for i in $(seq 8)
+		for i in $(test_seq 8)
 		do
 			git -C client checkout b$i &&
 			test_commit -C client b$i.c$j
@@ -205,7 +205,7 @@ test_expect_success 'do not send "have" with ancestors of commits that server AC
 
 	# fetch-pack should thus not send any more commits in the b1 branch, but
 	# should still send the others (in this test, just check b2).
-	for i in $(seq 0 8)
+	for i in $(test_seq 0 8)
 	do
 		have_not_sent b1.c$i
 	done &&
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index a73c55a47e..d1ccc22331 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -176,7 +176,7 @@ test_expect_success 'setup repos for change-while-negotiating test' '
 		git clone "http://127.0.0.1:$LIB_HTTPD_PORT/smart/repo" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
@@ -289,7 +289,7 @@ test_expect_success 'setup repos for fetching with ref-in-want tests' '
 		git clone "file://$REPO" "$LOCAL_PRISTINE" &&
 		cd "$LOCAL_PRISTINE" &&
 		git checkout -b side &&
-		for i in $(seq 1 33); do test_commit s$i; done &&
+		for i in $(test_seq 1 33); do test_commit s$i; done &&
 
 		# Add novel commits to upstream
 		git checkout master &&
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v4 3/6] tests: fix comment syntax in chainlint.sed for AIX sed

[Ævar Arnfjörð Bjarmason]

Change a comment in chainlint.sed to appease AIX sed, which would
previously print this error:

    sed:    # stash for later printing is not a recognized function

1. https://public-inbox.org/git/CAPig+cTTbU5HFMKgNyrxTp3+kcK46-Fn=4ZH6zDt1oQChAc3KA@mail.gmail.com/

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/chainlint.sed | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/t/chainlint.sed b/t/chainlint.sed
index 8544df38df..dcb4b333ed 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -131,9 +131,8 @@ b
 b
 
 :subshell
-# bare "(" line?
+# bare "(" line? -- stash for later printing
 /^[ 	]*([	]*$/ {
-	# stash for later printing
 	h
 	bnextline
 }
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v4 3/6] tests: fix comment syntax in chainlint.sed for AIX sed

[Eric Sunshine]

On Fri, Aug 24, 2018 at 11:20 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Change a comment in chainlint.sed to appease AIX sed, which would
> previously print this error:
>
>     sed:    # stash for later printing is not a recognized function
>
> 1. https://public-inbox.org/git/CAPig+cTTbU5HFMKgNyrxTp3+kcK46-Fn=4ZH6zDt1oQChAc3KA@mail.gmail.com/

It would have been nice for the commit message to explain that the
problem was caused by the comment being indented, since it's otherwise
not otherwise obvious to the reader without chasing the link to the
email exchange, but probably not worth a re-roll.

> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

FWIW,
Acked-by: Eric Sunshine <sunshine@sunshineco.com>

[PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

[Ævar Arnfjörð Bjarmason]

Improve the portability of chainlint by using shorter here-docs. On
AIX sed will complain about:

    sed: 0602-417 The label :hereslurp is greater than eight
    characters

This, in combination with the previous fix to this file makes
GIT_TEST_CHAIN_LINT=1 (which is the default) working again on AIX
without issues, and the "gmake check-chainlint" test also passes.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/chainlint.sed | 56 ++++++++++++++++++++++++-------------------------
 1 file changed, 28 insertions(+), 28 deletions(-)

diff --git a/t/chainlint.sed b/t/chainlint.sed
index dcb4b333ed..c80d2fad7a 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -97,11 +97,11 @@
 /<<[ 	]*[-\\']*[A-Za-z0-9_]/ {
 	s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
 	s/[ 	]*<<//
-	:hereslurp
+	:hered
 	N
 	/^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
 		s/\n.*$//
-		bhereslurp
+		bhered
 	}
 	s/^<[^>]*>//
 	s/\n.*$//
@@ -149,7 +149,7 @@ s/.*\n//
 
 :slurp
 # incomplete line "...\"
-/\\$/bincomplete
+/\\$/bicmplte
 # multi-line quoted string "...\n..."?
 /"/bdqstring
 # multi-line quoted string '...\n...'? (but not contraction in string "it's")
@@ -171,7 +171,7 @@ s/.*\n//
 	/"[^"]*#[^"]*"/!s/[ 	]#.*$//
 }
 # one-liner "case ... esac"
-/^[ 	]*case[ 	]*..*esac/bcheckchain
+/^[ 	]*case[ 	]*..*esac/bchkchn
 # multi-line "case ... esac"
 /^[ 	]*case[ 	]..*[ 	]in/bcase
 # multi-line "for ... done" or "while ... done"
@@ -200,32 +200,32 @@ s/.*\n//
 /^[ 	]*fi[ 	]*[<>|]/bdone
 /^[ 	]*fi[ 	]*)/bdone
 # nested one-liner "(...) &&"
-/^[ 	]*(.*)[ 	]*&&[ 	]*$/bcheckchain
+/^[ 	]*(.*)[ 	]*&&[ 	]*$/bchkchn
 # nested one-liner "(...)"
-/^[ 	]*(.*)[ 	]*$/bcheckchain
+/^[ 	]*(.*)[ 	]*$/bchkchn
 # nested one-liner "(...) >x" (or "2>x" or "<x" or "|x")
-/^[ 	]*(.*)[ 	]*[0-9]*[<>|]/bcheckchain
+/^[ 	]*(.*)[ 	]*[0-9]*[<>|]/bchkchn
 # nested multi-line "(...\n...)"
 /^[ 	]*(/bnest
 # multi-line "{...\n...}"
 /^[ 	]*{/bblock
 # closing ")" on own line -- exit subshell
-/^[ 	]*)/bclosesolo
+/^[ 	]*)/bclssolo
 # "$((...))" -- arithmetic expansion; not closing ")"
-/\$(([^)][^)]*))[^)]*$/bcheckchain
+/\$(([^)][^)]*))[^)]*$/bchkchn
 # "$(...)" -- command substitution; not closing ")"
-/\$([^)][^)]*)[^)]*$/bcheckchain
+/\$([^)][^)]*)[^)]*$/bchkchn
 # multi-line "$(...\n...)" -- command substitution; treat as nested subshell
 /\$([^)]*$/bnest
 # "=(...)" -- Bash array assignment; not closing ")"
-/=(/bcheckchain
+/=(/bchkchn
 # closing "...) &&"
 /)[ 	]*&&[ 	]*$/bclose
 # closing "...)"
 /)[ 	]*$/bclose
 # closing "...) >x" (or "2>x" or "<x" or "|x")
 /)[ 	]*[<>|]/bclose
-:checkchain
+:chkchn
 # mark suspect if line uses ";" internally rather than "&&" (but not ";" in a
 # string and not ";;" in one-liner "case...esac")
 /;/{
@@ -244,7 +244,7 @@ n
 bslurp
 
 # found incomplete line "...\" -- slurp up next line
-:incomplete
+:icmplte
 N
 s/\\\n//
 bslurp
@@ -282,11 +282,11 @@ bfolded
 :heredoc
 s/^\(.*\)<<[ 	]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</
 s/[ 	]*<<//
-:hereslurpsub
+:heredsub
 N
 /^<\([^>]*\)>.*\n[ 	]*\1[ 	]*$/!{
 	s/\n.*$//
-	bhereslurpsub
+	bheredsub
 }
 s/^<[^>]*>//
 s/\n.*$//
@@ -316,43 +316,43 @@ x
 # is 'done' or 'fi' cuddled with ")" to close subshell?
 /done.*)/bclose
 /fi.*)/bclose
-bcheckchain
+bchkchn
 
 # found nested multi-line "(...\n...)" -- pass through untouched
 :nest
 x
-:nestslurp
+:nstslurp
 n
 # closing ")" on own line -- stop nested slurp
-/^[ 	]*)/bnestclose
+/^[ 	]*)/bnstclose
 # comment -- not closing ")" if in comment
-/^[ 	]*#/bnestcontinue
+/^[ 	]*#/bnstcnt
 # "$((...))" -- arithmetic expansion; not closing ")"
-/\$(([^)][^)]*))[^)]*$/bnestcontinue
+/\$(([^)][^)]*))[^)]*$/bnstcnt
 # "$(...)" -- command substitution; not closing ")"
-/\$([^)][^)]*)[^)]*$/bnestcontinue
+/\$([^)][^)]*)[^)]*$/bnstcnt
 # closing "...)" -- stop nested slurp
-/)/bnestclose
-:nestcontinue
+/)/bnstclose
+:nstcnt
 x
-bnestslurp
-:nestclose
+bnstslurp
+:nstclose
 s/^/>>/
 # is it "))" which closes nested and parent subshells?
 /)[ 	]*)/bslurp
-bcheckchain
+bchkchn
 
 # found multi-line "{...\n...}" block -- pass through untouched
 :block
 x
 n
 # closing "}" -- stop block slurp
-/}/bcheckchain
+/}/bchkchn
 bblock
 
 # found closing ")" on own line -- drop "suspect" from final line of subshell
 # since that line legitimately lacks "&&" and exit subshell loop
-:closesolo
+:clssolo
 x
 s/?!AMP?!//
 p
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

[Eric Sunshine]

On Fri, Aug 24, 2018 at 11:20 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> Improve the portability of chainlint by using shorter here-docs. On
> AIX sed will complain about:
>
>     sed: 0602-417 The label :hereslurp is greater than eight
>     characters

Shortening the names makes them ugly and often unreadable. That's not
a complaint with this patch; just a general observation regarding
8-byte limitation with this platform's "sed" (and POSIX). See a few
suggested improvements below, but probably not worth a re-roll.

> This, in combination with the previous fix to this file makes
> GIT_TEST_CHAIN_LINT=1 (which is the default) working again on AIX
> without issues, and the "gmake check-chainlint" test also passes.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

FWIW,
Acked-by: Eric Sunshine <sunshine@sunshineco.com>

> diff --git a/t/chainlint.sed b/t/chainlint.sed
> -:checkchain
> +:chkchn

":chkchain"

>  # found incomplete line "...\" -- slurp up next line
> -:incomplete
> +:icmplte

":fold" (for "fold out NL")

>  # found nested multi-line "(...\n...)" -- pass through untouched
> -:nestslurp
> +:nstslurp

":nesteat"

> -:nestcontinue
> +:nstcnt

":nestcont"

> -:nestclose
> +:nstclose

":nestend" or ":endnest"

>  # found closing ")" on own line -- drop "suspect" from final line of subshell
>  # since that line legitimately lacks "&&" and exit subshell loop
> -:closesolo
> +:clssolo

":endsolo"

Re: [PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

[Ævar Arnfjörð Bjarmason]

On Fri, Aug 24 2018, Eric Sunshine wrote:

> On Fri, Aug 24, 2018 at 11:20 AM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> Improve the portability of chainlint by using shorter here-docs. On
>> AIX sed will complain about:
>>
>>     sed: 0602-417 The label :hereslurp is greater than eight
>>     characters
>
> Shortening the names makes them ugly and often unreadable. That's not
> a complaint with this patch; just a general observation regarding
> 8-byte limitation with this platform's "sed" (and POSIX). See a few
> suggested improvements below, but probably not worth a re-roll.
>
>> This, in combination with the previous fix to this file makes
>> GIT_TEST_CHAIN_LINT=1 (which is the default) working again on AIX
>> without issues, and the "gmake check-chainlint" test also passes.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> FWIW,
> Acked-by: Eric Sunshine <sunshine@sunshineco.com>
>
>> diff --git a/t/chainlint.sed b/t/chainlint.sed
>> -:checkchain
>> +:chkchn
>
> ":chkchain"
>
>>  # found incomplete line "...\" -- slurp up next line
>> -:incomplete
>> +:icmplte
>
> ":fold" (for "fold out NL")
>
>>  # found nested multi-line "(...\n...)" -- pass through untouched
>> -:nestslurp
>> +:nstslurp
>
> ":nesteat"
>
>> -:nestcontinue
>> +:nstcnt
>
> ":nestcont"
>
>> -:nestclose
>> +:nstclose
>
> ":nestend" or ":endnest"
>
>>  # found closing ")" on own line -- drop "suspect" from final line of subshell
>>  # since that line legitimately lacks "&&" and exit subshell loop
>> -:closesolo
>> +:clssolo
>
> ":endsolo"

I was meaning to get to this with a re-roll, but since this is already
in next & these label renames seem cosmetic, it seems better to do this
after 2.19, i.e. the compiler doesn't care about the specific names, but
shortening them to <=8 fixes the bug.

Re: [PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

[Eric Sunshine]

On Tue, Aug 28, 2018 at 4:14 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> On Fri, Aug 24 2018, Eric Sunshine wrote:
> > Shortening the names makes them ugly and often unreadable. That's not
> > a complaint with this patch; just a general observation regarding
> > 8-byte limitation with this platform's "sed" (and POSIX). See a few
> > suggested improvements below, but probably not worth a re-roll.
> >> +:clssolo
> > ":endsolo"
>
> I was meaning to get to this with a re-roll, but since this is already
> in next & these label renames seem cosmetic, it seems better to do this
> after 2.19, i.e. the compiler doesn't care about the specific names, but
> shortening them to <=8 fixes the bug.

Sounds fine. There's no rush. Thanks.

Re: [PATCH v4 4/6] tests: use shorter here-docs in chainlint.sed for AIX sed

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Improve the portability of chainlint by using shorter here-docs. On
> AIX sed will complain about:
>
>     sed: 0602-417 The label :hereslurp is greater than eight
>     characters

Remind me again not to forget doing s/here-doc/label/ on this patch
before queueing.  Other than that, looks good, together with 3/6 for
the the indented comment issue.

What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.  The ones marked with '.' do not appear in any of
the integration branches, but I am still holding onto them.

Git 2.19-rc2 is out.  Hopefully the tip of 'master' is more or less
identical to the final one without needing much updates.

You can find the changes described here in the integration branches
of the repositories listed at

    http://git-blame.blogspot.com/p/git-public-repositories.html

--------------------------------------------------
[Graduated to "master"]

* ab/portable (2018-08-27) 6 commits
  (merged to 'next' on 2018-08-27 at 37640e66ef)
 + tests: fix and add lint for non-portable grep --file
 + tests: fix version-specific portability issue in Perl JSON
 + tests: use shorter labels in chainlint.sed for AIX sed
 + tests: fix comment syntax in chainlint.sed for AIX sed
 + tests: fix and add lint for non-portable seq
 + tests: fix and add lint for non-portable head -c N
 (this branch is used by ab/portable-more.)

 Portability fix.


* ab/portable-more (2018-08-29) 2 commits
  (merged to 'next' on 2018-08-31 at d7b44993e4)
 + tests: fix non-portable iconv invocation
 + tests: fix non-portable "${var:-"str"}" construct
 (this branch uses ab/portable.)

 Portability fix.


* ds/commit-graph-lockfile-fix (2018-08-30) 1 commit
  (merged to 'next' on 2018-09-04 at 0876e6ddcc)
 + commit: don't use generation numbers if not needed

 "git merge-base" in 2.19-rc1 has performance regression when the
 (experimental) commit-graph feature is in use, which has been
 mitigated.


* en/directory-renames-nothanks (2018-08-30) 3 commits
  (merged to 'next' on 2018-08-31 at 91d663d688)
 + am: avoid directory rename detection when calling recursive merge machinery
 + merge-recursive: add ability to turn off directory rename detection
 + t3401: add another directory rename testcase for rebase and am

 Recent addition of "directory rename" heuristics to the
 merge-recursive backend makes the command susceptible to false
 positives and false negatives.  In the context of "git am -3",
 which does not know about surrounding unmodified paths and thus
 cannot inform the merge machinery about the full trees involved,
 this risk is particularly severe.  As such, the heuristic is
 disabled for "git am -3" to keep the machinery "more stupid but
 predictable".


* es/chain-lint-more (2018-08-29) 1 commit
  (merged to 'next' on 2018-08-31 at d456090b62)
 + chainlint: match "quoted" here-doc tags

 The test linter code has learned that the end of here-doc mark
 "EOF" can be quoted in a double-quote pair, not just in a
 single-quote pair.


* es/freebsd-iconv-portability (2018-08-31) 1 commit
  (merged to 'next' on 2018-09-04 at 52baa37dd7)
 + config.mak.uname: resolve FreeBSD iconv-related compilation warning

 Build fix.


* pw/rebase-i-author-script-fix (2018-08-07) 2 commits
  (merged to 'next' on 2018-08-31 at 7b9f485407)
 + sequencer: fix quoting in write_author_script
 + sequencer: handle errors from read_author_ident()

 Recent "git rebase -i" update started to write bogusly formatted
 author-script, with a matching broken reading code.  These are
 fixed.

--------------------------------------------------
[New Topics]

* ab/fetch-tags-noclobber (2018-08-31) 9 commits
 - fetch: stop clobbering existing tags without --force
 - fetch: document local ref updates with/without --force
 - push doc: correct lies about how push refspecs work
 - push doc: move mention of "tag <tag>" later in the prose
 - push doc: remove confusing mention of remote merger
 - fetch tests: add a test for clobbering tag behavior
 - push tests: use spaces in interpolated string
 - push tests: make use of unused $1 in test description
 - fetch: change "branch" to "reference" in --force -h output

 The rules used by "git push" and "git fetch" to determine if a ref
 can or cannot be updated were inconsistent; specifically, fetching
 to update existing tags were allowed even though tags are supposed
 to be unmoving anchoring points.  "git fetch" was taught to forbid
 updates to existing tags without the "--force" option.

 Will merge to and cook in 'next'.
 This is a backward incompatible change but in a good way; it may
 still need to be treated carefully.


* es/worktree-forced-ops-fix (2018-08-30) 9 commits
 - worktree: delete .git/worktrees if empty after 'remove'
 - worktree: teach 'remove' to override lock when --force given twice
 - worktree: teach 'move' to override lock when --force given twice
 - worktree: teach 'add' to respect --force for registered but missing path
 - worktree: disallow adding same path multiple times
 - worktree: prepare for more checks of whether path can become worktree
 - worktree: generalize delete_git_dir() to reduce code duplication
 - worktree: move delete_git_dir() earlier in file for upcoming new callers
 - worktree: don't die() in library function find_worktree()

 Various subcommands of "git worktree" take '--force' but did not
 behave sensibly, which has been corrected.


* jk/patch-corrupted-delta-fix (2018-08-30) 6 commits
 - t5303: use printf to generate delta bases
 - patch-delta: handle truncated copy parameters
 - patch-delta: consistently report corruption
 - patch-delta: fix oob read
 - t5303: test some corrupt deltas
 - test-delta: read input into a heap buffer

 Malformed or crafted data in packstream can make our code attempt
 to read or write past the allocated buffer and abort, instead of
 reporting an error, which has been fixed.

 Will merge to 'next'.


* jc/rebase-in-c-9-fixes (2018-09-04) 1 commit
 - rebase: re-add forgotten -k that stands for --keep-empty
 (this branch uses ag/rebase-i-in-c, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final.)


* jk/pack-objects-with-bitmap-fix (2018-09-04) 4 commits
 - pack-bitmap: drop "loaded" flag
 - traverse_bitmap_commit_list(): don't free result
 - t5310: test delta reuse with bitmaps
 - bitmap_has_sha1_in_uninteresting(): drop BUG check
 (this branch uses jk/pack-delta-reuse-with-bitmap.)

 Hotfix of the base topic.  It may not be a bad idea to squash these
 in when the next development cycle opens.


* js/rebase-i-autosquash-fix (2018-09-04) 2 commits
 - rebase -i: be careful to wrap up fixup/squash chains
 - rebase -i --autosquash: demonstrate a problem skipping the last squash

 "git rebase -i" did not clear the state files correctly when a run
 of "squash/fixup" is aborted and then the user manually amended the
 commit instead, which has been corrected.

 Will merge to 'next'.


* nd/bisect-show-list-fix (2018-09-04) 1 commit
 - bisect.c: make show_list() build again

 Debugging aid update.

 Will merge to and cook in 'next'.


* nd/optim-reading-index-v4 (2018-09-04) 1 commit
 - read-cache.c: optimize reading index format v4

 The v4 format of the index file uses prefix compression to store
 the pathnames to save file size.  The codepath to read such a file
 has been optimized.

 Will merge to and cook in 'next'.


* sg/doc-trace-appends (2018-09-04) 1 commit
 - Documentation/git.txt: clarify that GIT_TRACE=/path appends

 Docfix.

 Will merge to 'next'.

--------------------------------------------------
[Stalled]

* sl/commit-dry-run-with-short-output-fix (2018-07-30) 4 commits
 . commit: fix exit code when doing a dry run
 . wt-status: teach wt_status_collect about merges in progress
 . wt-status: rename commitable to committable
 . t7501: add coverage for flags which imply dry runs

 "git commit --dry-run" gave a correct exit status even during a
 conflict resolution toward a merge, but it did not with the
 "--short" option, which has been corrected.

 Seems to break 7512, 3404 and 7060 in 'pu'.


* ma/wrapped-info (2018-05-28) 2 commits
 - usage: prefix all lines in `vreportf()`, not just the first
 - usage: extract `prefix_suffix_lines()` from `advise()`

 An attempt to help making multi-line messages fed to warning(),
 error(), and friends more easily translatable.

 Will discard and wait for a cleaned-up rewrite.
 cf. <20180529213957.GF7964@sigill.intra.peff.net>


* hn/bisect-first-parent (2018-04-21) 1 commit
 - bisect: create 'bisect_flags' parameter in find_bisection()
 (this branch is used by tb/bisect-first-parent.)

 Preliminary code update to allow passing more flags down the
 bisection codepath in the future.

 We do not add random code that does not have real users to our
 codebase, so let's have it wait until such a real code materializes
 before too long.


* av/fsmonitor-updates (2018-01-04) 6 commits
 - fsmonitor: use fsmonitor data in `git diff`
 - fsmonitor: remove debugging lines from t/t7519-status-fsmonitor.sh
 - fsmonitor: make output of test-dump-fsmonitor more concise
 - fsmonitor: update helper tool, now that flags are filled later
 - fsmonitor: stop inline'ing mark_fsmonitor_valid / _invalid
 - dir.c: update comments to match argument name

 Code clean-up on fsmonitor integration, plus optional utilization
 of the fsmonitor data in diff-files.

 Waiting for an update.
 cf. <alpine.DEB.2.21.1.1801042335130.32@MININT-6BKU6QN.europe.corp.microsoft.com>


* pb/bisect-helper-2 (2018-07-23) 8 commits
 - t6030: make various test to pass GETTEXT_POISON tests
 - bisect--helper: `bisect_start` shell function partially in C
 - bisect--helper: `get_terms` & `bisect_terms` shell function in C
 - bisect--helper: `bisect_next_check` shell function in C
 - bisect--helper: `check_and_set_terms` shell function in C
 - wrapper: move is_empty_file() and rename it as is_empty_or_missing_file()
 - bisect--helper: `bisect_write` shell function in C
 - bisect--helper: `bisect_reset` shell function in C

 Expecting a reroll.
 cf. <0102015f5e5ee171-f30f4868-886f-47a1-a4e4-b4936afc545d-000000@eu-west-1.amazonses.com>

 I just rebased the topic to a newer base as it did not build
 standalone with the base I originally queued the topic on, but
 otherwise there is no update to address any of the review comments
 in the thread above---we are still waiting for a reroll.


* jk/drop-ancient-curl (2017-08-09) 5 commits
 - http: #error on too-old curl
 - curl: remove ifdef'd code never used with curl >=7.19.4
 - http: drop support for curl < 7.19.4
 - http: drop support for curl < 7.16.0
 - http: drop support for curl < 7.11.1

 Some code in http.c that has bitrot is being removed.

 Expecting a reroll.


* mk/use-size-t-in-zlib (2017-08-10) 1 commit
 . zlib.c: use size_t for size

 The wrapper to call into zlib followed our long tradition to use
 "unsigned long" for sizes of regions in memory, which have been
 updated to use "size_t".

 Needs resurrecting by making sure the fix is good and still applies
 (or adjusted to today's codebase).

--------------------------------------------------
[Cooking]

* ds/format-commit-graph-docs (2018-08-21) 2 commits
 - commit-graph.txt: improve formatting for asciidoc
 - Docs: Add commit-graph tech docs to Makefile

 Design docs for the commit-graph machinery is now made into HTML as
 well as text.


* jk/diff-rendered-docs (2018-08-31) 5 commits
 - doc/Makefile: drop doc-diff worktree and temporary files on "make clean"
 - doc-diff: add --clean mode to remove temporary working gunk
 - doc-diff: fix non-portable 'man' invocation
 - doc-diff: always use oids inside worktree
  (merged to 'next' on 2018-08-22 at dd7a2b71cd)
 + SubmittingPatches: mention doc-diff

 Dev doc update.

 Will merge to 'next'.


* jk/pack-delta-reuse-with-bitmap (2018-08-21) 6 commits
  (merged to 'next' on 2018-08-22 at fc50b59dab)
 + pack-objects: reuse on-disk deltas for thin "have" objects
 + pack-bitmap: save "have" bitmap from walk
 + t/perf: add perf tests for fetches from a bitmapped server
 + t/perf: add infrastructure for measuring sizes
 + t/perf: factor out percent calculations
 + t/perf: factor boilerplate out of test_perf
 (this branch is used by jk/pack-objects-with-bitmap-fix.)

 When creating a thin pack, which allows objects to be made into a
 delta against another object that is not in the resulting pack but
 is known to be present on the receiving end, the code learned to
 take advantage of the reachability bitmap; this allows the server
 to send a delta against a base beyond the "boundary" commit.

 Will cook in 'next'.


* jk/rev-list-stdin-noop-is-ok (2018-08-22) 1 commit
  (merged to 'next' on 2018-08-27 at d5916f7bc1)
 + rev-list: make empty --stdin not an error

 "git rev-list --stdin </dev/null" used to be an error; it now shows
 no output without an error.  "git rev-list --stdin --default HEAD"
 still falls back to the given default when nothing is given on the
 standard input.

 Will cook in 'next'.


* js/rebase-in-c-5.5-work-with-rebase-i-in-c (2018-08-29) 2 commits
 - builtin rebase: prepare for builtin rebase -i
 - Merge branch 'ag/rebase-i-in-c' into js/rebase-in-c-5.5-work-with-rebase-i-in-c
 (this branch is used by jc/rebase-in-c-9-fixes and pk/rebase-in-c-6-final; uses ag/rebase-i-in-c, pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts and pk/rebase-in-c-5-test.)

 "rebase" that has been rewritten learns the new calling convention
 used by "rebase -i" that was rewritten in C, tying the loose end
 between two GSoC topics that stomped on each other's toes.


* jk/trailer-fixes (2018-08-23) 8 commits
  (merged to 'next' on 2018-08-27 at 93b671b8c6)
 + append_signoff: use size_t for string offsets
 + sequencer: ignore "---" divider when parsing trailers
 + pretty, ref-filter: format %(trailers) with no_divider option
 + interpret-trailers: allow suppressing "---" divider
 + interpret-trailers: tighten check for "---" patch boundary
 + trailer: pass process_trailer_opts to trailer_info_get()
 + trailer: use size_t for iterating trailer list
 + trailer: use size_t for string offsets

 "git interpret-trailers" and its underlying machinery had a buggy
 code that attempted to ignore patch text after commit log message,
 which triggered in various codepaths that will always get the log
 message alone and never get such an input.

 Will cook in 'next'.


* tg/rerere-doc-updates (2018-08-29) 2 commits
  (merged to 'next' on 2018-08-31 at ce4fef1a97)
 + rerere: add note about files with existing conflict markers
 + rerere: mention caveat about unmatched conflict markers
 (this branch uses tg/rerere.)

 Clarify a part of technical documentation for rerere.

 Will cook in 'next'.


* ds/commit-graph-tests (2018-08-29) 1 commit
 - commit-graph: define GIT_TEST_COMMIT_GRAPH

 We can now optionally run tests with commit-graph enabled.

 Will merge to 'next'.


* jk/cocci (2018-08-29) 9 commits
  (merged to 'next' on 2018-08-31 at 914b4f17ce)
 + show_dirstat: simplify same-content check
 + read-cache: use oideq() in ce_compare functions
 + convert hashmap comparison functions to oideq()
 + convert "hashcmp() != 0" to "!hasheq()"
 + convert "oidcmp() != 0" to "!oideq()"
 + convert "hashcmp() == 0" to hasheq()
 + convert "oidcmp() == 0" to oideq()
 + introduce hasheq() and oideq()
 + coccinelle: use <...> for function exclusion

 spatch transformation to replace boolean uses of !hashcmp() to
 newly introduced oideq() is added, and applied, to regain
 performance lost due to support of multiple hash algorithms.

 Will cook in 'next'.


* js/add-i-coalesce-after-editing-hunk (2018-08-28) 1 commit
 - add -p: coalesce hunks before testing applicability

 Applicability check after a patch is edited in a "git add -i/p"
 session has been improved.

 Will hold.
 cf. <e5b2900a-0558-d3bf-8ea1-d526b078bbc2@talktalk.net>


* rs/mailinfo-format-flowed (2018-08-29) 1 commit
  (merged to 'next' on 2018-08-31 at 9e8b92176c)
 + mailinfo: support format=flowed

 "git mailinfo" used in "git am" learned to make a best-effort
 recovery of a patch corrupted by MUA that sends text/plain with
 format=flawed option.

 Will cook in 'next'.


* sb/submodule-move-head-with-corruption (2018-08-28) 2 commits
 - submodule.c: warn about missing submodule git directories
 - t2013: add test for missing but active submodule

 A corner case in switching to a branch that needs to "check out" a
 submodule is handled a bit better now.

 Expecting a reroll.


* tg/conflict-marker-size (2018-08-29) 1 commit
  (merged to 'next' on 2018-08-31 at 12099161f0)
 + .gitattributes: add conflict-marker-size for relevant files

 Developer aid.

 Will cook in 'next'.


* ts/doc-build-manpage-xsl-quietly (2018-08-29) 1 commit
  (merged to 'next' on 2018-08-31 at 7527e0f8d3)
 + Documentation/Makefile: make manpage-base-url.xsl generation quieter

 Build tweak.

 Will cook in 'next'.


* bp/checkout-new-branch-optim (2018-08-16) 1 commit
  (merged to 'next' on 2018-08-27 at e69bfd115f)
 + checkout: optimize "git checkout -b <new_branch>"

 "git checkout -b newbranch [HEAD]" should not have to do as much as
 checking out a commit different from HEAD.  An attempt is made to
 optimize this special case.

 Will cook in 'next'.


* ao/submodule-wo-gitmodules-checked-out (2018-08-14) 7 commits
 - submodule: support reading .gitmodules even when it's not checked out
 - t7506: clean up .gitmodules properly before setting up new scenario
 - submodule: use the 'submodule--helper config' command
 - submodule--helper: add a new 'config' subcommand
 - t7411: be nicer to future tests and really clean things up
 - submodule: factor out a config_set_in_gitmodules_file_gently function
 - submodule: add a print_config_from_gitmodules() helper

 The submodule support has been updated to read from the blob at
 HEAD:.gitmodules when the .gitmodules file is missing from the
 working tree.

 Expecting a reroll.

 I find the design a bit iffy in that our usual "missing in the
 working tree?  let's use the latest blob" fallback is to take it
 from the index, not from the HEAD.


* bw/submodule-name-to-dir (2018-08-10) 2 commits
  (merged to 'next' on 2018-08-22 at c17f83be24)
 + submodule: munge paths to submodule git directories
 + submodule: create helper to build paths to submodule gitdirs

 In modern repository layout, the real body of a cloned submodule
 repository is held in .git/modules/ of the superproject, indexed by
 the submodule name.  URLencode the submodule name before computing
 the name of the directory to make sure they form a flat namespace.

 Expecting further work on the topic.
 cf. <CAGZ79kYnbjaPoWdda0SM_-_X77mVyYC7JO61OV8nm2yj3Q1OvQ@mail.gmail.com>


* cc/delta-islands (2018-08-16) 7 commits
  (merged to 'next' on 2018-08-27 at cf3d7bd93f)
 + pack-objects: move 'layer' into 'struct packing_data'
 + pack-objects: move tree_depth into 'struct packing_data'
 + t5320: tests for delta islands
 + repack: add delta-islands support
 + pack-objects: add delta-islands support
 + pack-objects: refactor code into compute_layer_order()
 + Add delta-islands.{c,h}

 Lift code from GitHub to restrict delta computation so that an
 object that exists in one fork is not made into a delta against
 another object that does not appear in the same forked repository.

 Will cook in 'next'.


* md/filter-trees (2018-09-04) 7 commits
 - list-objects-filter: implement filter tree:0
 - list-objects-filter: use BUG rather than die
 - revision: mark non-user-given objects instead
 - rev-list: handle missing tree objects properly
 - list-objects: always parse trees gently
 - list-objects: refactor to process_tree_contents
 - list-objects: store common func args in struct

 The "rev-list --filter" feature learned to exclude all trees via
 "tree:0" filter.

 Will merge to 'next'.


* ng/status-i-short-for-ignored (2018-08-09) 1 commit
 - status: -i shorthand for --ignored command line option

 "git status --ignored" gained a shorthand "git status -i".

 What's the list opinion on this one?  It is Meh to me, but
 obviously the author cared enough to write a patch, so...


* pk/rebase-in-c-2-basic (2018-08-10) 11 commits
 - builtin rebase: support `git rebase <upstream> <switch-to>`
 - builtin rebase: only store fully-qualified refs in `options.head_name`
 - builtin rebase: start a new rebase only if none is in progress
 - builtin rebase: support --force-rebase
 - builtin rebase: try to fast forward when possible
 - builtin rebase: require a clean worktree
 - builtin rebase: support the `verbose` and `diffstat` options
 - builtin rebase: support --quiet
 - builtin rebase: handle the pre-rebase hook (and add --no-verify)
 - builtin rebase: support `git rebase --onto A...B`
 - builtin rebase: support --onto
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final; uses pk/rebase-in-c.)


* pk/rebase-in-c-3-acts (2018-08-10) 7 commits
 - builtin rebase: stop if `git am` is in progress
 - builtin rebase: actions require a rebase in progress
 - builtin rebase: support --edit-todo and --show-current-patch
 - builtin rebase: support --quit
 - builtin rebase: support --abort
 - builtin rebase: support --skip
 - builtin rebase: support --continue
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final; uses pk/rebase-in-c and pk/rebase-in-c-2-basic.)


* pk/rebase-in-c-4-opts (2018-08-10) 18 commits
 - builtin rebase: support --root
 - builtin rebase: add support for custom merge strategies
 - builtin rebase: support `fork-point` option
 - merge-base --fork-point: extract libified function
 - builtin rebase: support --rebase-merges[=[no-]rebase-cousins]
 - builtin rebase: support `--allow-empty-message` option
 - builtin rebase: support `--exec`
 - builtin rebase: support `--autostash` option
 - builtin rebase: support `-C` and `--whitespace=<type>`
 - builtin rebase: support `--gpg-sign` option
 - builtin rebase: support `--autosquash`
 - builtin rebase: support `keep-empty` option
 - builtin rebase: support `ignore-date` option
 - builtin rebase: support `ignore-whitespace` option
 - builtin rebase: support --committer-date-is-author-date
 - builtin rebase: support --rerere-autoupdate
 - builtin rebase: support --signoff
 - builtin rebase: allow selecting the rebase "backend"
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final; uses pk/rebase-in-c, pk/rebase-in-c-2-basic and pk/rebase-in-c-3-acts.)


* pk/rebase-in-c-5-test (2018-08-10) 6 commits
 - builtin rebase: error out on incompatible option/mode combinations
 - builtin rebase: use no-op editor when interactive is "implied"
 - builtin rebase: show progress when connected to a terminal
 - builtin rebase: fast-forward to onto if it is a proper descendant
 - builtin rebase: optionally pass custom reflogs to reset_head()
 - builtin rebase: optionally auto-detect the upstream
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c and pk/rebase-in-c-6-final; uses pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts and pk/rebase-in-c-4-opts.)


* pk/rebase-in-c-6-final (2018-08-29) 1 commit
 - rebase: default to using the builtin rebase
 (this branch is used by jc/rebase-in-c-9-fixes; uses ag/rebase-i-in-c, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts and pk/rebase-in-c-5-test.)


* ps/stash-in-c (2018-08-31) 20 commits
 - stash: replace all `write-tree` child processes with API calls
 - stash: optimize `get_untracked_files()` and `check_changes()`
 - stash: convert `stash--helper.c` into `stash.c`
 - stash: convert save to builtin
 - stash: make push -q quiet
 - stash: convert push to builtin
 - stash: convert create to builtin
 - stash: convert store to builtin
 - stash: mention options in `show` synopsis
 - stash: convert show to builtin
 - stash: convert list to builtin
 - stash: convert pop to builtin
 - stash: convert branch to builtin
 - stash: convert drop and clear to builtin
 - stash: convert apply to builtin
 - stash: add tests for `git stash show` config
 - stash: rename test cases to be more descriptive
 - stash: update test cases conform to coding guidelines
 - stash: improve option parsing test coverage
 - sha1-name.c: add `get_oidf()` which acts like `get_oid()`


* nd/clone-case-smashing-warning (2018-08-17) 1 commit
  (merged to 'next' on 2018-08-22 at eedae40a8d)
 + clone: report duplicate entries on case-insensitive filesystems

 Running "git clone" against a project that contain two files with
 pathnames that differ only in cases on a case insensitive
 filesystem would result in one of the files lost because the
 underlying filesystem is incapable of holding both at the same
 time.  An attempt is made to detect such a case and warn.

 Will cook in 'next'.


* nd/unpack-trees-with-cache-tree (2018-08-27) 8 commits
  (merged to 'next' on 2018-08-27 at b1d841d034)
 + Document update for nd/unpack-trees-with-cache-tree
  (merged to 'next' on 2018-08-22 at 138b902673)
 + cache-tree: verify valid cache-tree in the test suite
 + unpack-trees: add missing cache invalidation
 + unpack-trees: reuse (still valid) cache-tree from src_index
 + unpack-trees: reduce malloc in cache-tree walk
 + unpack-trees: optimize walking same trees with cache-tree
 + unpack-trees: add performance tracing
 + trace.h: support nested performance tracing

 The unpack_trees() API used in checking out a branch and merging
 walks one or more trees along with the index.  When the cache-tree
 in the index tells us that we are walking a tree whose flattened
 contents is known (i.e. matches a span in the index), as linearly
 scanning a span in the index is much more efficient than having to
 open tree objects recursively and listing their entries, the walk
 can be optimized, which is done in this topic.

 Will cook in 'next'.


* sb/range-diff-colors (2018-08-20) 11 commits
  (merged to 'next' on 2018-08-22 at eb7ed4fca3)
 + range-diff: indent special lines as context
 + range-diff: make use of different output indicators
 + diff.c: add --output-indicator-{new, old, context}
 + diff.c: rewrite emit_line_0 more understandably
 + diff.c: omit check for line prefix in emit_line_0
 + diff: use emit_line_0 once per line
 + diff.c: add set_sign to emit_line_0
 + diff.c: reorder arguments for emit_line_ws_markup
 + diff.c: simplify caller of emit_line_0
 + t3206: add color test for range-diff --dual-color
 + test_decode_color: understand FAINT and ITALIC

 The color output support for recently introduced "range-diff"
 command got tweaked a bit.

 Will cook in 'next'.


* sg/t1404-update-ref-test-timeout (2018-08-01) 1 commit
  (merged to 'next' on 2018-08-22 at f3cd37b5ea)
 + t1404: increase core.packedRefsTimeout to avoid occasional test failure

 An attempt to unflake a test a bit.

 Will cook in 'next'.


* pw/add-p-select (2018-07-26) 4 commits
 - add -p: optimize line selection for short hunks
 - add -p: allow line selection to be inverted
 - add -p: select modified lines correctly
 - add -p: select individual hunk lines

 "git add -p" interactive interface learned to let users choose
 individual added/removed lines to be used in the operation, instead
 of accepting or rejecting a whole hunk.

 Will hold.
 cf. <d622a95b-7302-43d4-4ec9-b2cf3388c653@talktalk.net>
 I found the feature to be hard to explain, and may result in more
 end-user complaints, but let's see.


* ds/commit-graph-with-grafts (2018-08-21) 8 commits
 - commit-graph: close_commit_graph before shallow walk
 - commit-graph: not compatible with uninitialized repo
 - commit-graph: not compatible with grafts
 - commit-graph: not compatible with replace objects
 - test-repository: properly init repo
 - commit-graph: update design document
 - refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callback
 - refs.c: migrate internal ref iteration to pass thru repository argument

 The recently introduced commit-graph auxiliary data is incompatible
 with mechanisms such as replace & grafts that "breaks" immutable
 nature of the object reference relationship.  Disable optimizations
 based on its use (and updating existing commit-graph) when these
 incompatible features are in use in the repository.

 Replaced with a newer version.


* ds/reachable (2018-08-28) 19 commits
  (merged to 'next' on 2018-08-28 at b1634b371d)
 + commit-reach: correct accidental #include of C file
  (merged to 'next' on 2018-08-22 at 17f3275afb)
 + commit-reach: use can_all_from_reach
 + commit-reach: make can_all_from_reach... linear
 + commit-reach: replace ref_newer logic
 + test-reach: test commit_contains
 + test-reach: test can_all_from_reach_with_flags
 + test-reach: test reduce_heads
 + test-reach: test get_merge_bases_many
 + test-reach: test is_descendant_of
 + test-reach: test in_merge_bases
 + test-reach: create new test tool for ref_newer
 + commit-reach: move can_all_from_reach_with_flags
 + upload-pack: generalize commit date cutoff
 + upload-pack: refactor ok_to_give_up()
 + upload-pack: make reachable() more generic
 + commit-reach: move commit_contains from ref-filter
 + commit-reach: move ref_newer from remote.c
 + commit.h: remove method declarations
 + commit-reach: move walk methods from commit.c

 The code for computing history reachability has been shuffled,
 obtained a bunch of new tests to cover them, and then being
 improved.

 Will cook in 'next'.


* es/format-patch-interdiff (2018-07-23) 6 commits
  (merged to 'next' on 2018-08-31 at 63927e0227)
 + format-patch: allow --interdiff to apply to a lone-patch
 + log-tree: show_log: make commentary block delimiting reusable
 + interdiff: teach show_interdiff() to indent interdiff
 + format-patch: teach --interdiff to respect -v/--reroll-count
 + format-patch: add --interdiff option to embed diff in cover letter
 + format-patch: allow additional generated content in make_cover_letter()
 (this branch is used by es/format-patch-rangediff.)

 "git format-patch" learned a new "--interdiff" option to explain
 the difference between this version and the previous atttempt in
 the cover letter (or after the tree-dashes as a comment).

 Will cook in 'next'.


* es/format-patch-rangediff (2018-08-14) 10 commits
  (merged to 'next' on 2018-08-31 at 65627afece)
 + format-patch: allow --range-diff to apply to a lone-patch
 + format-patch: add --creation-factor tweak for --range-diff
 + format-patch: teach --range-diff to respect -v/--reroll-count
 + format-patch: extend --range-diff to accept revision range
 + format-patch: add --range-diff option to embed diff in cover letter
 + range-diff: relieve callers of low-level configuration burden
 + range-diff: publish default creation factor
 + range-diff: respect diff_option.file rather than assuming 'stdout'
 + Merge branch 'es/format-patch-interdiff' into es/format-patch-rangediff
 + Merge branch 'js/range-diff' into es/format-patch-rangediff
 (this branch uses es/format-patch-interdiff.)

 "git format-patch" learned a new "--range-diff" option to explain
 the difference between this version and the previous attempt in
 the cover letter (or after the tree-dashes as a comment).

 Will cook in 'next'.


* jn/gc-auto (2018-07-17) 3 commits
 - gc: do not return error for prior errors in daemonized mode
 - gc: exit with status 128 on failure
 - gc: improve handling of errors reading gc.log

 "gc --auto" ended up calling exit(-1) upon error, which has been
 corrected to use exit(1).  Also the error reporting behaviour when
 daemonized has been updated to exit with zero status when stopping
 due to a previously discovered error (which implies there is no
 point running gc to improve the situation); we used to exit with
 failure in such a case.

 What's the donness of this one?
 cf. <20180717201348.GD26218@sigill.intra.peff.net>


* sb/submodule-update-in-c (2018-08-14) 7 commits
  (merged to 'next' on 2018-08-17 at 23c81e5ff7)
 + submodule--helper: introduce new update-module-mode helper
 + submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree
 + builtin/submodule--helper: factor out method to update a single submodule
 + builtin/submodule--helper: store update_clone information in a struct
 + builtin/submodule--helper: factor out submodule updating
 + git-submodule.sh: rename unused variables
 + git-submodule.sh: align error reporting for update mode to use path

 "git submodule update" is getting rewritten piece-by-piece into C.

 Will cook in 'next'.


* tg/rerere (2018-08-06) 11 commits
  (merged to 'next' on 2018-08-17 at 919a958cdc)
 + rerere: recalculate conflict ID when unresolved conflict is committed
 + rerere: teach rerere to handle nested conflicts
 + rerere: return strbuf from handle path
 + rerere: factor out handle_conflict function
 + rerere: only return whether a path has conflicts or not
 + rerere: fix crash with files rerere can't handle
 + rerere: add documentation for conflict normalization
 + rerere: mark strings for translation
 + rerere: wrap paths in output in sq
 + rerere: lowercase error messages
 + rerere: unify error messages when read_cache fails
 (this branch is used by tg/rerere-doc-updates.)

 Fixes to "git rerere" corner cases, especially when conflict
 markers cannot be parsed in the file.

 Will cook in 'next'.


* ag/rebase-i-in-c (2018-08-29) 20 commits
 - rebase -i: move rebase--helper modes to rebase--interactive
 - rebase -i: remove git-rebase--interactive.sh
 - rebase--interactive2: rewrite the submodes of interactive rebase in C
 - rebase -i: implement the main part of interactive rebase as a builtin
 - rebase -i: rewrite init_basic_state() in C
 - rebase -i: rewrite write_basic_state() in C
 - rebase -i: rewrite the rest of init_revisions_and_shortrevisions() in C
 - rebase -i: implement the logic to initialize $revisions in C
 - rebase -i: remove unused modes and functions
 - rebase -i: rewrite complete_action() in C
 - t3404: todo list with commented-out commands only aborts
 - sequencer: change the way skip_unnecessary_picks() returns its result
 - sequencer: refactor append_todo_help() to write its message to a buffer
 - rebase -i: rewrite checkout_onto() in C
 - rebase -i: rewrite setup_reflog_action() in C
 - sequencer: add a new function to silence a command, except if it fails
 - rebase -i: rewrite the edit-todo functionality in C
 - editor: add a function to launch the sequence editor
 - rebase -i: rewrite append_todo_help() in C
 - sequencer: make three functions and an enum from sequencer.c public
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c and pk/rebase-in-c-6-final.)

 Rewrite of the remaining "rebase -i" machinery in C.


* lt/date-human (2018-07-09) 1 commit
 - Add 'human' date format

 A new date format "--date=human" that morphs its output depending
 on how far the time is from the current time has been introduced.
 "--date=auto" can be used to use this new format when the output is
 goint to the pager or to the terminal and otherwise the default
 format.


* pk/rebase-in-c (2018-08-06) 3 commits
 - builtin/rebase: support running "git rebase <upstream>"
 - rebase: refactor common shell functions into their own file
 - rebase: start implementing it as a builtin
 (this branch is used by jc/rebase-in-c-9-fixes, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final.)

 Rewrite of the "rebase" machinery in C.


* jk/branch-l-1-repurpose (2018-08-30) 2 commits
  (merged to 'next' on 2018-08-31 at cfa73bbfcb)
 + doc/git-branch: remove obsolete "-l" references
  (merged to 'next' on 2018-08-08 at d2a08dd08e)
 + branch: make "-l" a synonym for "--list"

 Updated plan to repurpose the "-l" option to "git branch".

 Will cook in 'next'.


* ds/multi-pack-index (2018-08-20) 33 commits
  (merged to 'next' on 2018-08-21 at d15e8cadd4)
 + pack-objects: consider packs in multi-pack-index
 + midx: test a few commands that use get_all_packs
 + treewide: use get_all_packs
 + packfile: add all_packs list
 + midx: fix bug that skips midx with alternates
 + midx: stop reporting garbage
 + midx: mark bad packed objects
 + multi-pack-index: store local property
 + multi-pack-index: provide more helpful usage info
 + Sync 'ds/multi-pack-index' to v2.19.0-rc0
  (merged to 'next' on 2018-08-08 at 1a56c52967)
 + midx: clear midx on repack
 + packfile: skip loading index if in multi-pack-index
 + midx: prevent duplicate packfile loads
 + midx: use midx in approximate_object_count
 + midx: use existing midx when writing new one
 + midx: use midx in abbreviation calculations
 + midx: read objects from multi-pack-index
 + config: create core.multiPackIndex setting
 + midx: write object offsets
 + midx: write object id fanout chunk
 + midx: write object ids in a chunk
 + midx: sort and deduplicate objects from packfiles
 + midx: read pack names into array
 + multi-pack-index: write pack names in chunk
 + multi-pack-index: read packfile list
 + packfile: generalize pack directory list
 + t5319: expand test data
 + multi-pack-index: load into memory
 + midx: write header information to lockfile
 + multi-pack-index: add 'write' verb
 + multi-pack-index: add builtin
 + multi-pack-index: add format details
 + multi-pack-index: add design document

 When there are too many packfiles in a repository (which is not
 recommended), looking up an object in these would require
 consulting many pack .idx files; a new mechanism to have a single
 file that consolidates all of these .idx files is introduced.

 Will cook in 'next'.

--------------------------------------------------
[Discarded]

* am/sequencer-author-script-fix (2018-07-18) 1 commit
 . sequencer.c: terminate the last line of author-script properly

 The author-script that records the author information created by
 the sequencer machinery lacked the closing single quote on the last
 entry.

 Superseded by another topic.


* jc/push-cas-opt-comment (2018-08-01) 1 commit
 . push: comment on a funny unbalanced option help

 Code clarification.

 Superseded by another topic.


* cc/remote-odb (2018-08-02) 9 commits
 . Documentation/config: add odb.<name>.promisorRemote
 . t0410: test fetching from many promisor remotes
 . Use odb.origin.partialclonefilter instead of core.partialclonefilter
 . Use remote_odb_get_direct() and has_remote_odb()
 . remote-odb: add remote_odb_reinit()
 . remote-odb: implement remote_odb_get_many_direct()
 . remote-odb: implement remote_odb_get_direct()
 . Add initial remote odb support
 . fetch-object: make functions return an error code

 Implement lazy fetches of missing objects to complement the
 experimental partial clone feature.

 Ejected; seems to break existing repositories that use partialclone
 repository extension.


* jh/structured-logging (2018-08-28) 26 commits
 . SQUASH??? spatch fix
 . structured-logging: add config data facility
 . structured-logging: t0420 tests for interacitve child_summary
 . structured-logging: t0420 tests for child process detail events
 . structured-logging: add child process classification
 . structured-logging: add detail-events for child processes
 . structured-logging: add structured logging to remote-curl
 . structured-logging: t0420 tests for aux-data
 . structured-logging: add aux-data for size of sparse-checkout file
 . structured-logging: add aux-data for index size
 . structured-logging: add aux-data facility
 . structured-logging: t0420 tests for timers
 . structured-logging: add timer around preload_index
 . structured-logging: add timer around wt-status functions
 . structured-logging: add timer around do_write_index
 . structured-logging: add timer around do_read_index
 . structured-logging: add timer facility
 . structured-logging: add detail-event for lazy_init_name_hash
 . structured-logging: add detail-event facility
 . structured-logging: t0420 basic tests
 . structured-logging: set sub_command field for checkout command
 . structured-logging: set sub_command field for branch command
 . structured-logging: add session-id to log events
 . structured-logging: add structured logging framework
 . structured-logging: add STRUCTURED_LOGGING=1 to Makefile
 . structured-logging: design document

 Being rerolled with an updated tracing API.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Eric Sunshine]

On Tue, Sep 4, 2018 at 6:36 PM Junio C Hamano <gitster@pobox.com> wrote:
> * es/worktree-forced-ops-fix (2018-08-30) 9 commits
>  - worktree: delete .git/worktrees if empty after 'remove'
>  - worktree: teach 'remove' to override lock when --force given twice
>  - worktree: teach 'move' to override lock when --force given twice
>  - worktree: teach 'add' to respect --force for registered but missing path
>  - worktree: disallow adding same path multiple times
>  - worktree: prepare for more checks of whether path can become worktree
>  - worktree: generalize delete_git_dir() to reduce code duplication
>  - worktree: move delete_git_dir() earlier in file for upcoming new callers
>  - worktree: don't die() in library function find_worktree()
>
>  Various subcommands of "git worktree" take '--force' but did not
>  behave sensibly, which has been corrected.

This description mischaracterizes what these changes are about. The
primary intent of this series is to fix a bug in which the same path
can be registered under multiple worktree entries.

As for --force, it worked perfectly fine for the couple git-worktree
subcommands which accepted it. The patches in this series dealing
with --force are merely extending it to other subcommands or to other
use-cases.

So, perhaps rewrite this description like this:

    Fix a bug in which the same path could be registered under
    multiple worktree entries if the patch was missing (for instance,
    was removed manually).

    Also, as a convenience, expand the number of cases in which
    --force is applicable.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> This description mischaracterizes what these changes are about.

Thanks for a replacement text; very much appreciated.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Eric Sunshine]

On Wed, Sep 5, 2018 at 5:14 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Tue, Sep 4, 2018 at 6:36 PM Junio C Hamano <gitster@pobox.com> wrote:
> > * es/worktree-forced-ops-fix (2018-08-30) 9 commits
> This description mischaracterizes what these changes are about. [...]
>
> So, perhaps rewrite this description like this:
>
>     Fix a bug in which the same path could be registered under
>     multiple worktree entries if the patch was missing (for instance,
>     was removed manually).

Bah, typo: s/patch/path/

>     Also, as a convenience, expand the number of cases in which
>     --force is applicable.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Eric Sunshine]

On Tue, Sep 4, 2018 at 6:36 PM Junio C Hamano <gitster@pobox.com> wrote:
> * es/worktree-forced-ops-fix (2018-08-30) 9 commits
>  - worktree: delete .git/worktrees if empty after 'remove'
>  - worktree: teach 'remove' to override lock when --force given twice
>  - worktree: teach 'move' to override lock when --force given twice
>  - worktree: teach 'add' to respect --force for registered but missing path
>  - worktree: disallow adding same path multiple times
>  - worktree: prepare for more checks of whether path can become worktree
>  - worktree: generalize delete_git_dir() to reduce code duplication
>  - worktree: move delete_git_dir() earlier in file for upcoming new callers
>  - worktree: don't die() in library function find_worktree()
>
>  Various subcommands of "git worktree" take '--force' but did not
>  behave sensibly, which has been corrected.

This series is missing a patch[1] from Peff which he wanted placed at
the end of the series. It was probably overlooked since he embedded
it as a reply in that thread rather than sending it as a standalone
patch.

[1]: https://public-inbox.org/git/20180830075431.GF11944@sigill.intra.peff.net/

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Jeff King]

On Wed, Sep 05, 2018 at 05:50:04AM -0400, Eric Sunshine wrote:

> On Tue, Sep 4, 2018 at 6:36 PM Junio C Hamano <gitster@pobox.com> wrote:
> > * es/worktree-forced-ops-fix (2018-08-30) 9 commits
> >  - worktree: delete .git/worktrees if empty after 'remove'
> >  - worktree: teach 'remove' to override lock when --force given twice
> >  - worktree: teach 'move' to override lock when --force given twice
> >  - worktree: teach 'add' to respect --force for registered but missing path
> >  - worktree: disallow adding same path multiple times
> >  - worktree: prepare for more checks of whether path can become worktree
> >  - worktree: generalize delete_git_dir() to reduce code duplication
> >  - worktree: move delete_git_dir() earlier in file for upcoming new callers
> >  - worktree: don't die() in library function find_worktree()
> >
> >  Various subcommands of "git worktree" take '--force' but did not
> >  behave sensibly, which has been corrected.
> 
> This series is missing a patch[1] from Peff which he wanted placed at
> the end of the series. It was probably overlooked since he embedded
> it as a reply in that thread rather than sending it as a standalone
> patch.
> 
> [1]: https://public-inbox.org/git/20180830075431.GF11944@sigill.intra.peff.net/

Yeah, I'm not sure which is easier for Junio. I figured by replying
inline, it makes it easy to pick up on top of the others (since it
really does depend on them and should be in the same topic). But it is
easier to overlook as "meh, just more discussion".

Either way, yes, I'd be happy to see that patch on top.

-Peff

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> [1]: https://public-inbox.org/git/20180830075431.GF11944@sigill.intra.peff.net/
>
> Yeah, I'm not sure which is easier for Junio. I figured by replying
> inline, it makes it easy to pick up on top of the others (since it
> really does depend on them and should be in the same topic). But it is
> easier to overlook as "meh, just more discussion".
>
> Either way, yes, I'd be happy to see that patch on top.

Thanks.

jc/rebase-in-c-9-fixes, was Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Johannes Schindelin]

Hi Junio,

On Tue, 4 Sep 2018, Junio C Hamano wrote:

> * jc/rebase-in-c-9-fixes (2018-09-04) 1 commit
>  - rebase: re-add forgotten -k that stands for --keep-empty
>  (this branch uses ag/rebase-i-in-c, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final.)

Quite frankly, I'd rather you pick up my two v2 series. I promised to take
over from Pratik, who is busy with exams, and I did. I even announced as
much in that mail where I pointed out the missing short option originally.

So there is no need to do add-on patches, as if the `rebase-in-c` patch
series had already been accepted into `next`.

Ciao,
Dscho

Re: jc/rebase-in-c-9-fixes, was Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Tue, 4 Sep 2018, Junio C Hamano wrote:
>
>> * jc/rebase-in-c-9-fixes (2018-09-04) 1 commit
>>  - rebase: re-add forgotten -k that stands for --keep-empty
>>  (this branch uses ag/rebase-i-in-c, js/rebase-in-c-5.5-work-with-rebase-i-in-c, pk/rebase-in-c, pk/rebase-in-c-2-basic, pk/rebase-in-c-3-acts, pk/rebase-in-c-4-opts, pk/rebase-in-c-5-test and pk/rebase-in-c-6-final.)
>
> Quite frankly, I'd rather you pick up my two v2 series. I promised to take
> over from Pratik, who is busy with exams, and I did. I even announced as
> much in that mail where I pointed out the missing short option originally.

Yup, I'm planning to replace the large pile of patches with the new
ones you'd send once the final is done.  The topic above is just a
stop-gap measure to keep the tip of 'pu' testable, to be dropped
when the series it fixes is replaced with a corrected one.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Duy Nguyen]

On Wed, Sep 5, 2018 at 6:35 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Junio C Hamano <gitster@pobox.com> writes:
>
> > Here are the topics that have been cooking.  Commits prefixed with
> > '-' are only in 'pu' (proposed updates) while commits prefixed with
> > '+' are in 'next'.  The ones marked with '.' do not appear in any of
> > the integration branches, but I am still holding onto them.
> >
> > Git 2.19-rc2 is out.  Hopefully the tip of 'master' is more or less
> > identical to the final one without needing much updates.
>
> By the way, linux-gcc job of TravisCI seems to have been unhappy
> lately all the way down to 'master'.  It fails split-index tests,
> which may or may not be new breakage.
>
>     https://travis-ci.org/git/git/jobs/424552273
>
> If this is a recent regression, we may want to revert a few commits,
> but I do not offhand recall us having touched the spilt-index part
> of the code during this cycle.

I can't reproduce it here (with either 64 or 32 bit builds on x86).
Not denying the problem, just a quick update. I'll need more time,
maybe over weekend to have a closer look.
-- 
Duy

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[SZEDER Gábor]

On Wed, Sep 05, 2018 at 06:48:06PM +0200, Duy Nguyen wrote:
> On Wed, Sep 5, 2018 at 6:35 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Junio C Hamano <gitster@pobox.com> writes:
> >
> > > Here are the topics that have been cooking.  Commits prefixed with
> > > '-' are only in 'pu' (proposed updates) while commits prefixed with
> > > '+' are in 'next'.  The ones marked with '.' do not appear in any of
> > > the integration branches, but I am still holding onto them.
> > >
> > > Git 2.19-rc2 is out.  Hopefully the tip of 'master' is more or less
> > > identical to the final one without needing much updates.
> >
> > By the way, linux-gcc job of TravisCI seems to have been unhappy
> > lately all the way down to 'master'.  It fails split-index tests,
> > which may or may not be new breakage.
> >
> >     https://travis-ci.org/git/git/jobs/424552273
> >
> > If this is a recent regression, we may want to revert a few commits,
> > but I do not offhand recall us having touched the spilt-index part
> > of the code during this cycle.

It's not a regression, it's as old as the split index feature itself.

> I can't reproduce it here (with either 64 or 32 bit builds on x86).
> Not denying the problem, just a quick update. I'll need more time,
> maybe over weekend to have a closer look.

I have the patches almost ready, only the commit messages need some
touching up.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Ævar Arnfjörð Bjarmason]

On Tue, Sep 04 2018, Junio C Hamano wrote:

> Git 2.19-rc2 is out.  Hopefully the tip of 'master' is more or less
> identical to the final one without needing much updates.
>
> You can find the changes described here in the integration branches
> of the repositories listed at
>
>     http://git-blame.blogspot.com/p/git-public-repositories.html
>
> --------------------------------------------------
> [Graduated to "master"]
>
> * ab/portable (2018-08-27) 6 commits
>   (merged to 'next' on 2018-08-27 at 37640e66ef)
>  + tests: fix and add lint for non-portable grep --file
>  + tests: fix version-specific portability issue in Perl JSON
>  + tests: use shorter labels in chainlint.sed for AIX sed
>  + tests: fix comment syntax in chainlint.sed for AIX sed
>  + tests: fix and add lint for non-portable seq
>  + tests: fix and add lint for non-portable head -c N
>  (this branch is used by ab/portable-more.)

I recently gained access to a Solaris 10 SPARC (5.10) box and discovered
that the chainlint.sed implementation in 2.19.0 has more sed portability
issues.

First, whoever implemented the /bin/sed on Solaris apparently read the
POSIX requirements for a max length label of 8 to mean that 8 characters
should include the colon, so a bunch of things fail because of that, but
are fixed with a shorter 7 character label.

Then GIT_TEST_CHAIN_LINT=1 still fails because 878f988350 ("t/test-lib:
teach --chain-lint to detect broken &&-chains in subshells", 2018-07-11)
added a "grep -q" invocation. The /bin/grep on that version of Solaris
doesn't have -q. We fixed a similar issue way back in 80700fde91
("t/t1304: make a second colon optional in the mask ACL check",
2010-03-15) by redirecting to /dev/null instead.

A bunch of other tests in the test suite rely on "grep -q", but nothing
as central as chainlint, so it makes everything break. Do we want to
away with "grep -q" entirely because of old Solaris /bin/grep?

At this point those familiar with Solaris are screaming ("why are you
using anything in /bin!"). Okey fine, but it mostly worked before, so
are we OK with breaking it? "Mostly" here is "test suite would fail
20-30 tests for various reasons, but at least no failures in test-lib.sh
and the like".

However, if as config.mak.uname does we run the tests with
PATH=/usr/xpg6/bin:/usr/xpg4/bin:$PATH, at this point sed is fine with 8
character labels, but starts complaining about this (also in
chainlint.sed):

    sed: Too many commands, last: s/\n//

As with other sed issues I fixed recently in chainlint.sed this one is
just the tip of the iceberg. Once you "fix" one (just remove it, I have
no idea how to rewrite it) others appear.

I was hoping this would just be a Solaris 10 issue, but it's also an
issue in Solaris 11 (5.11 11.3).

So, do we want to chase this down or just do this?:

    diff --git a/Makefile b/Makefile
    index 5a969f5..f125dc5 100644
    --- a/Makefile
    +++ b/Makefile
    @@ -2602,6 +2602,9 @@ endif
     ifdef TEST_GIT_INDEX_VERSION
            @echo TEST_GIT_INDEX_VERSION=\''$(subst ','\'',$(subst ','\'',$(TEST_GIT_INDEX_VERSION)))'\' >>$@+
     endif
    +ifdef GIT_TEST_CHAIN_LINT
    +       @echo GIT_TEST_CHAIN_LINT=\''$(subst ','\'',$(subst ','\'',$(GIT_TEST_CHAIN_LINT)))'\' >>$@+
    +endif
            @if cmp $@+ $@ >/dev/null 2>&1; then $(RM) $@+; else mv $@+ $@; fi

     ### Detect Python interpreter path changes
    diff --git a/config.mak.uname b/config.mak.uname
    index e47af72..2b02a2b 100644
    --- a/config.mak.uname
    +++ b/config.mak.uname
    @@ -163,6 +163,10 @@ ifeq ($(uname_S),SunOS)
            INSTALL = /usr/ucb/install
            TAR = gtar
            BASIC_CFLAGS += -D__EXTENSIONS__ -D__sun__
    +       # t/chainlint.sed is hopelessly broken all known (tested
    +       # Solaris 10 & 11) versions of Solaris, both /bin/sed and
    +       # /usr/xpg4/bin/sed
    +       GIT_TEST_CHAIN_LINT = 0
     endif
     ifeq ($(uname_O),Cygwin)
            ifeq ($(shell expr "$(uname_R)" : '1\.[1-6]\.'),4)

If I could wave a magic wand I'd say "let's just rewrite chainlint.sed
in perl, at least that's portable", but that's a lot of effort (and I
doubt Eric wants to). It slightly sucks to not have chainlint on
Solaris, but it would also suck to revert chainlint.sed back to 2.18.0
(there were some big improvements). So I think the patch above is the
best way forward, especially since we're on rc2. What do you think?

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Eric Sunshine]

On Wed, Sep 5, 2018 at 4:29 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
> I recently gained access to a Solaris 10 SPARC (5.10) box and discovered
> that the chainlint.sed implementation in 2.19.0 has more sed portability
> issues.
>
> First, whoever implemented the /bin/sed on Solaris apparently read the
> POSIX requirements for a max length label of 8 to mean that 8 characters
> should include the colon, so a bunch of things fail because of that, but
> are fixed with a shorter 7 character label.

I'm pretty sure that Solaris 'sed' predates POSIX by a good bit, but
that's neither here nor there.

> Then GIT_TEST_CHAIN_LINT=1 still fails because 878f988350 ("t/test-lib:
> teach --chain-lint to detect broken &&-chains in subshells", 2018-07-11)
> added a "grep -q" invocation. The /bin/grep on that version of Solaris
> doesn't have -q.

I knew that '-q' was potentially problematic on some platforms, so I
checked and saw that existing tests were already using it, thus went
ahead and used it. Dropping '-q' here and redirecting stderr to
/dev/null is a perfectly fine alternative.

> We fixed a similar issue way back in 80700fde91
> ("t/t1304: make a second colon optional in the mask ACL check",
> 2010-03-15) by redirecting to /dev/null instead.
>
> A bunch of other tests in the test suite rely on "grep -q", but nothing
> as central as chainlint, so it makes everything break. Do we want to
> away with "grep -q" entirely because of old Solaris /bin/grep?

I count 132 instances in existing tests (though, I may have missed some).

> At this point those familiar with Solaris are screaming ("why are you
> using anything in /bin!"). Okey fine, but it mostly worked before, so
> are we OK with breaking it? "Mostly" here is "test suite would fail
> 20-30 tests for various reasons, but at least no failures in test-lib.sh
> and the like".
>
> However, if as config.mak.uname does we run the tests with
> PATH=/usr/xpg6/bin:/usr/xpg4/bin:$PATH, at this point sed is fine with 8
> character labels [...]

So, if you run the tests via 'make test' (or whatever), then you get
/usr/xpg?/bin in PATH, but if you run an individual test script (and
haven't set your PATH appropriately), then you encounter the problems
you report above?

> [...] but starts complaining about this (also in
> chainlint.sed):
>
>     sed: Too many commands, last: s/\n//

Erm. Any additional context to help narrow this down?

>     diff --git a/config.mak.uname b/config.mak.uname
>     @@ -163,6 +163,10 @@ ifeq ($(uname_S),SunOS)
>             BASIC_CFLAGS += -D__EXTENSIONS__ -D__sun__
>     +       # t/chainlint.sed is hopelessly broken all known (tested
>     +       # Solaris 10 & 11) versions of Solaris, both /bin/sed and
>     +       # /usr/xpg4/bin/sed
>     +       GIT_TEST_CHAIN_LINT = 0
>      endif
>
> It slightly sucks to not have chainlint on
> Solaris, but it would also suck to revert chainlint.sed back to 2.18.0
> (there were some big improvements). So I think the patch above is the
> best way forward, especially since we're on rc2. What do you think?

Keeping in mind that the main goal of 'chainlint' is to prevent _new_
breakage from entering the test suite, disabling 'chainlint' on
Solaris is an entirely reasonable way forward. In present day, it
seems quite unlikely that we'll see someone develop new tests on
Solaris, so having 'chainlint' disabled there isn't likely to be a big
deal. Moreover, if someone does write a new test on Solaris which has
a broken &&-chain in a subshell, that breakage will be caught very
quickly once the test is run by anyone on Linux or MacOS (or Windows
or BSD or AIX), so it's not like the broken test will make it into the
project undetected.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Ævar Arnfjörð Bjarmason]

On Wed, Sep 05 2018, Eric Sunshine wrote:

> On Wed, Sep 5, 2018 at 4:29 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> I recently gained access to a Solaris 10 SPARC (5.10) box and discovered
>> that the chainlint.sed implementation in 2.19.0 has more sed portability
>> issues.
>>
>> First, whoever implemented the /bin/sed on Solaris apparently read the
>> POSIX requirements for a max length label of 8 to mean that 8 characters
>> should include the colon, so a bunch of things fail because of that, but
>> are fixed with a shorter 7 character label.
>
> I'm pretty sure that Solaris 'sed' predates POSIX by a good bit, but
> that's neither here nor there.
>
>> Then GIT_TEST_CHAIN_LINT=1 still fails because 878f988350 ("t/test-lib:
>> teach --chain-lint to detect broken &&-chains in subshells", 2018-07-11)
>> added a "grep -q" invocation. The /bin/grep on that version of Solaris
>> doesn't have -q.
>
> I knew that '-q' was potentially problematic on some platforms, so I
> checked and saw that existing tests were already using it, thus went
> ahead and used it. Dropping '-q' here and redirecting stderr to
> /dev/null is a perfectly fine alternative.
>
>> We fixed a similar issue way back in 80700fde91
>> ("t/t1304: make a second colon optional in the mask ACL check",
>> 2010-03-15) by redirecting to /dev/null instead.
>>
>> A bunch of other tests in the test suite rely on "grep -q", but nothing
>> as central as chainlint, so it makes everything break. Do we want to
>> away with "grep -q" entirely because of old Solaris /bin/grep?
>
> I count 132 instances in existing tests (though, I may have missed some).
>
>> At this point those familiar with Solaris are screaming ("why are you
>> using anything in /bin!"). Okey fine, but it mostly worked before, so
>> are we OK with breaking it? "Mostly" here is "test suite would fail
>> 20-30 tests for various reasons, but at least no failures in test-lib.sh
>> and the like".
>>
>> However, if as config.mak.uname does we run the tests with
>> PATH=/usr/xpg6/bin:/usr/xpg4/bin:$PATH, at this point sed is fine with 8
>> character labels [...]
>
> So, if you run the tests via 'make test' (or whatever), then you get
> /usr/xpg?/bin in PATH, but if you run an individual test script (and
> haven't set your PATH appropriately), then you encounter the problems
> you report above?

You need to manually set the PATH before running the tests, the
SANE_TOOL_PATH just affects git-sh-setup. We should probably fix that,
i.e. now if you compile git our shellscripts will use the fixed paths,
but not the tests.

>> [...] but starts complaining about this (also in
>> chainlint.sed):
>>
>>     sed: Too many commands, last: s/\n//
>
> Erm. Any additional context to help narrow this down?

I tried playing around with it a bit, but honestly don't understand
enough of this advanced sed syntax to make any headway, it's complaining
about e.g. the "check for multi-line double-quoted string" rule, but
then if you remove the s/\n// from there it complains about the next
rule in that format.

If you want to dig into this yourself I think the best way forward is
the last two paragraphs of
https://public-inbox.org/git/20180824152016.20286-1-avarab@gmail.com/

>>     diff --git a/config.mak.uname b/config.mak.uname
>>     @@ -163,6 +163,10 @@ ifeq ($(uname_S),SunOS)
>>             BASIC_CFLAGS += -D__EXTENSIONS__ -D__sun__
>>     +       # t/chainlint.sed is hopelessly broken all known (tested
>>     +       # Solaris 10 & 11) versions of Solaris, both /bin/sed and
>>     +       # /usr/xpg4/bin/sed
>>     +       GIT_TEST_CHAIN_LINT = 0
>>      endif
>>
>> It slightly sucks to not have chainlint on
>> Solaris, but it would also suck to revert chainlint.sed back to 2.18.0
>> (there were some big improvements). So I think the patch above is the
>> best way forward, especially since we're on rc2. What do you think?
>
> Keeping in mind that the main goal of 'chainlint' is to prevent _new_
> breakage from entering the test suite, disabling 'chainlint' on
> Solaris is an entirely reasonable way forward. In present day, it
> seems quite unlikely that we'll see someone develop new tests on
> Solaris, so having 'chainlint' disabled there isn't likely to be a big
> deal. Moreover, if someone does write a new test on Solaris which has
> a broken &&-chain in a subshell, that breakage will be caught very
> quickly once the test is run by anyone on Linux or MacOS (or Windows
> or BSD or AIX), so it's not like the broken test will make it into the
> project undetected.

Since writing my initial message I see that the CSW package packaging
git on Solaris just uses GNU userland:
https://sourceforge.net/p/gar/code/HEAD/tree/csw/mgar/pkg/git/trunk/Makefile

I.e. it puts /opt/csw/gnu in its $PATH, so this whole thing is probably
a non-issue. I.e. if the near-as-you-can-get "official" package just
compiles git against GNU tooling, perhaps we should stop caring about
Solaris-native tooling.

That does also mean that something like my patch above isn't OK, since
we shouldn't be disabling GIT_TEST_CHAIN_LINT based on the uname, if it
then turns out that GNU tooling is being used.

So I think for the purposes of v2.19.0-rc2..v2.19.0 it's fine to just
leave this as it is in master right now.

It's somewhat of a shame that we're drifting to not supporing some of
these more obscure POSIX systems "natively" though, but maybe that's the
right move at this point, and maybe it isn't.

When I was porting C code to Solaris ~10 years ago I'd often get SunCC
turning up some interesting issues that were *real* issues, same with
compiling with IBM xlc on AIX.

Now when I'm re-visiting these systems much later I've yet to turn up
some issue like that, just boring compatibility issues with their old
tooling. E.g. one outstanding issue is that some test of ours fail on
AIX because we use "readlink", but that command isn't in POSIX (only the
C function is), so AIX doesn't implement it.

SunCC used to be ahead of GCC & Clang when it came to certain classes of
warnings, but e.g. now everything it complains about is because it
doesn't understand C as well, e.g. we have quite a few compile warnings
due to code like this, which it claims is unreachable (but isn't):
https://github.com/git/git/blob/v2.19.0-rc2/read-cache.c#L950-L955

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Ævar Arnfjörð Bjarmason]

On Wed, Sep 05 2018, Ævar Arnfjörð Bjarmason wrote:

> On Wed, Sep 05 2018, Eric Sunshine wrote:
>
>> On Wed, Sep 5, 2018 at 4:29 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>>> I recently gained access to a Solaris 10 SPARC (5.10) box and discovered
>>> that the chainlint.sed implementation in 2.19.0 has more sed portability
>>> issues.
>>>
>>> First, whoever implemented the /bin/sed on Solaris apparently read the
>>> POSIX requirements for a max length label of 8 to mean that 8 characters
>>> should include the colon, so a bunch of things fail because of that, but
>>> are fixed with a shorter 7 character label.
>>
>> I'm pretty sure that Solaris 'sed' predates POSIX by a good bit, but
>> that's neither here nor there.
>>
>>> Then GIT_TEST_CHAIN_LINT=1 still fails because 878f988350 ("t/test-lib:
>>> teach --chain-lint to detect broken &&-chains in subshells", 2018-07-11)
>>> added a "grep -q" invocation. The /bin/grep on that version of Solaris
>>> doesn't have -q.
>>
>> I knew that '-q' was potentially problematic on some platforms, so I
>> checked and saw that existing tests were already using it, thus went
>> ahead and used it. Dropping '-q' here and redirecting stderr to
>> /dev/null is a perfectly fine alternative.
>>
>>> We fixed a similar issue way back in 80700fde91
>>> ("t/t1304: make a second colon optional in the mask ACL check",
>>> 2010-03-15) by redirecting to /dev/null instead.
>>>
>>> A bunch of other tests in the test suite rely on "grep -q", but nothing
>>> as central as chainlint, so it makes everything break. Do we want to
>>> away with "grep -q" entirely because of old Solaris /bin/grep?
>>
>> I count 132 instances in existing tests (though, I may have missed some).
>>
>>> At this point those familiar with Solaris are screaming ("why are you
>>> using anything in /bin!"). Okey fine, but it mostly worked before, so
>>> are we OK with breaking it? "Mostly" here is "test suite would fail
>>> 20-30 tests for various reasons, but at least no failures in test-lib.sh
>>> and the like".
>>>
>>> However, if as config.mak.uname does we run the tests with
>>> PATH=/usr/xpg6/bin:/usr/xpg4/bin:$PATH, at this point sed is fine with 8
>>> character labels [...]
>>
>> So, if you run the tests via 'make test' (or whatever), then you get
>> /usr/xpg?/bin in PATH, but if you run an individual test script (and
>> haven't set your PATH appropriately), then you encounter the problems
>> you report above?
>
> You need to manually set the PATH before running the tests, the
> SANE_TOOL_PATH just affects git-sh-setup. We should probably fix that,
> i.e. now if you compile git our shellscripts will use the fixed paths,
> but not the tests.
>
>>> [...] but starts complaining about this (also in
>>> chainlint.sed):
>>>
>>>     sed: Too many commands, last: s/\n//
>>
>> Erm. Any additional context to help narrow this down?
>
> I tried playing around with it a bit, but honestly don't understand
> enough of this advanced sed syntax to make any headway, it's complaining
> about e.g. the "check for multi-line double-quoted string" rule, but
> then if you remove the s/\n// from there it complains about the next
> rule in that format.
>
> If you want to dig into this yourself I think the best way forward is
> the last two paragraphs of
> https://public-inbox.org/git/20180824152016.20286-1-avarab@gmail.com/
>
>>>     diff --git a/config.mak.uname b/config.mak.uname
>>>     @@ -163,6 +163,10 @@ ifeq ($(uname_S),SunOS)
>>>             BASIC_CFLAGS += -D__EXTENSIONS__ -D__sun__
>>>     +       # t/chainlint.sed is hopelessly broken all known (tested
>>>     +       # Solaris 10 & 11) versions of Solaris, both /bin/sed and
>>>     +       # /usr/xpg4/bin/sed
>>>     +       GIT_TEST_CHAIN_LINT = 0
>>>      endif
>>>
>>> It slightly sucks to not have chainlint on
>>> Solaris, but it would also suck to revert chainlint.sed back to 2.18.0
>>> (there were some big improvements). So I think the patch above is the
>>> best way forward, especially since we're on rc2. What do you think?
>>
>> Keeping in mind that the main goal of 'chainlint' is to prevent _new_
>> breakage from entering the test suite, disabling 'chainlint' on
>> Solaris is an entirely reasonable way forward. In present day, it
>> seems quite unlikely that we'll see someone develop new tests on
>> Solaris, so having 'chainlint' disabled there isn't likely to be a big
>> deal. Moreover, if someone does write a new test on Solaris which has
>> a broken &&-chain in a subshell, that breakage will be caught very
>> quickly once the test is run by anyone on Linux or MacOS (or Windows
>> or BSD or AIX), so it's not like the broken test will make it into the
>> project undetected.
>
> Since writing my initial message I see that the CSW package packaging
> git on Solaris just uses GNU userland:
> https://sourceforge.net/p/gar/code/HEAD/tree/csw/mgar/pkg/git/trunk/Makefile
>
> I.e. it puts /opt/csw/gnu in its $PATH, so this whole thing is probably
> a non-issue. I.e. if the near-as-you-can-get "official" package just
> compiles git against GNU tooling, perhaps we should stop caring about
> Solaris-native tooling.
>
> That does also mean that something like my patch above isn't OK, since
> we shouldn't be disabling GIT_TEST_CHAIN_LINT based on the uname, if it
> then turns out that GNU tooling is being used.
>
> So I think for the purposes of v2.19.0-rc2..v2.19.0 it's fine to just
> leave this as it is in master right now.
>
> It's somewhat of a shame that we're drifting to not supporing some of
> these more obscure POSIX systems "natively" though, but maybe that's the
> right move at this point, and maybe it isn't.
>
> When I was porting C code to Solaris ~10 years ago I'd often get SunCC
> turning up some interesting issues that were *real* issues, same with
> compiling with IBM xlc on AIX.
>
> Now when I'm re-visiting these systems much later I've yet to turn up
> some issue like that, just boring compatibility issues with their old
> tooling. E.g. one outstanding issue is that some test of ours fail on
> AIX because we use "readlink", but that command isn't in POSIX (only the
> C function is), so AIX doesn't implement it.
>
> SunCC used to be ahead of GCC & Clang when it came to certain classes of
> warnings, but e.g. now everything it complains about is because it
> doesn't understand C as well, e.g. we have quite a few compile warnings
> due to code like this, which it claims is unreachable (but isn't):
> https://github.com/git/git/blob/v2.19.0-rc2/read-cache.c#L950-L955

Correction. It is still useful for something:
https://public-inbox.org/git/87a7ly1djb.fsf@evledraar.gmail.com/

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Torsten Bögershausen]

On Sat, Nov 24, 2018 at 08:33:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, Sep 05 2018, Ævar Arnfjörð Bjarmason wrote:
> 
> > On Wed, Sep 05 2018, Eric Sunshine wrote:

[]

> > SunCC used to be ahead of GCC & Clang when it came to certain classes of
> > warnings, but e.g. now everything it complains about is because it
> > doesn't understand C as well, e.g. we have quite a few compile warnings
> > due to code like this, which it claims is unreachable (but isn't):
> > https://github.com/git/git/blob/v2.19.0-rc2/read-cache.c#L950-L955
> 

Wait a second - even if the compiler claims something (wrong)...
there a still 1+1/2 questions from my side:


int verify_path(const char *path, unsigned mode)
{
	char c;
	     ^
	/* Q1: should  "c" be initialized like this: */
	char c = *path;
        
	if (has_dos_drive_prefix(path))
		return 0;

	goto inside;
	^^^^^^^^^^^^ /* Q2: and why do we need the "goto" here ? */
	for (;;) {
		if (!c)
			return 1;
		if (is_dir_sep(c)) {
inside:

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Torsten Bögershausen]

On Sun, Nov 25, 2018 at 05:28:35AM +0100, Torsten Bögershausen wrote:
> On Sat, Nov 24, 2018 at 08:33:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
> > 
> > On Wed, Sep 05 2018, Ævar Arnfjörð Bjarmason wrote:
> > 
> > > On Wed, Sep 05 2018, Eric Sunshine wrote:
> 
> []
> 
> > > SunCC used to be ahead of GCC & Clang when it came to certain classes of
> > > warnings, but e.g. now everything it complains about is because it
> > > doesn't understand C as well, e.g. we have quite a few compile warnings
> > > due to code like this, which it claims is unreachable (but isn't):
> > > https://github.com/git/git/blob/v2.19.0-rc2/read-cache.c#L950-L955
> > 
> 
> Wait a second - even if the compiler claims something (wrong)...
> there a still 1+1/2 questions from my side:
> 
> 
> int verify_path(const char *path, unsigned mode)
> {
> 	char c;
> 	     ^
> 	/* Q1: should  "c" be initialized like this: */
> 	char c = *path;
>         
> 	if (has_dos_drive_prefix(path))
> 		return 0;
> 
> 	goto inside;
> 	^^^^^^^^^^^^ /* Q2: and why do we need the "goto" here ? */
> 	for (;;) {
> 		if (!c)
> 			return 1;
> 		if (is_dir_sep(c)) {
> inside:

After some re-reading,
I think that the "goto inside" was just hard to read....

Out of interest:
would the following make the compiler happy ?


diff --git a/read-cache.c b/read-cache.c
index 49add63fe1..d574d58b9d 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -951,17 +951,15 @@ static int verify_dotfile(const char *rest, unsigned mode)
 
 int verify_path(const char *path, unsigned mode)
 {
-	char c;
+	char c = *path ? '/' : '\0';
 
 	if (has_dos_drive_prefix(path))
 		return 0;
 
-	goto inside;
 	for (;;) {
 		if (!c)
 			return 1;
 		if (is_dir_sep(c)) {
-inside:
 			if (protect_hfs) {
 				if (is_hfs_dotgit(path))
 					return 0;

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 25 2018, Torsten Bögershausen wrote:

> On Sun, Nov 25, 2018 at 05:28:35AM +0100, Torsten Bögershausen wrote:
>> On Sat, Nov 24, 2018 at 08:33:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> >
>> > On Wed, Sep 05 2018, Ævar Arnfjörð Bjarmason wrote:
>> >
>> > > On Wed, Sep 05 2018, Eric Sunshine wrote:
>>
>> []
>>
>> > > SunCC used to be ahead of GCC & Clang when it came to certain classes of
>> > > warnings, but e.g. now everything it complains about is because it
>> > > doesn't understand C as well, e.g. we have quite a few compile warnings
>> > > due to code like this, which it claims is unreachable (but isn't):
>> > > https://github.com/git/git/blob/v2.19.0-rc2/read-cache.c#L950-L955
>> >
>>
>> Wait a second - even if the compiler claims something (wrong)...
>> there a still 1+1/2 questions from my side:
>>
>>
>> int verify_path(const char *path, unsigned mode)
>> {
>> 	char c;
>> 	     ^
>> 	/* Q1: should  "c" be initialized like this: */
>> 	char c = *path;
>>
>> 	if (has_dos_drive_prefix(path))
>> 		return 0;
>>
>> 	goto inside;
>> 	^^^^^^^^^^^^ /* Q2: and why do we need the "goto" here ? */
>> 	for (;;) {
>> 		if (!c)
>> 			return 1;
>> 		if (is_dir_sep(c)) {
>> inside:
>
> After some re-reading,
> I think that the "goto inside" was just hard to read....
>
> Out of interest:
> would the following make the compiler happy ?
>
>
> diff --git a/read-cache.c b/read-cache.c
> index 49add63fe1..d574d58b9d 100644
> --- a/read-cache.c
> +++ b/read-cache.c
> @@ -951,17 +951,15 @@ static int verify_dotfile(const char *rest, unsigned mode)
>
>  int verify_path(const char *path, unsigned mode)
>  {
> -	char c;
> +	char c = *path ? '/' : '\0';
>
>  	if (has_dos_drive_prefix(path))
>  		return 0;
>
> -	goto inside;
>  	for (;;) {
>  		if (!c)
>  			return 1;
>  		if (is_dir_sep(c)) {
> -inside:
>  			if (protect_hfs) {
>  				if (is_hfs_dotgit(path))
>  					return 0;

I haven't tested (it's tedious) but yes, I can tell you SunCC won't
whine about this. I've only seen its unreachability detector get
confused about goto inside a loop, not the the sort of code you've
replaced this with.

We should not be appeasing these old compiler warnings in cases where
they're wrong. You can check out the CI output I linked to to see 10-20
cases in the codebase where SunCC is wrong about unreliability.

Whether we should just fix this for its own sake is another question.

Re: What's cooking in git.git (Sep 2018, #01; Tue, 4)

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Wed, Sep 5, 2018 at 4:29 AM Ævar Arnfjörð Bjarmason <avarab@gmail.com> wrote:
>> I recently gained access to a Solaris 10 SPARC (5.10) box and discovered
>> that the chainlint.sed implementation in 2.19.0 has more sed portability
>> issues.
>>
>> First, whoever implemented the /bin/sed on Solaris apparently read the
>> POSIX requirements for a max length label of 8 to mean that 8 characters
>> should include the colon, so a bunch of things fail because of that, but
>> are fixed with a shorter 7 character label.
>
> I'm pretty sure that Solaris 'sed' predates POSIX by a good bit, but
> that's neither here nor there.

I thought that we long time ago declared that it is a lost cause to
use Solaris with only tools in /bin and /usr/bin and instead depend
on /usr/xpg[46]/bin; did something happen recently to make us
reconsider the position?  If not, let's not waste time worrying
about /bin/sed over there.

[PATCH v4 5/6] tests: fix version-specific portability issue in Perl JSON

[Ævar Arnfjörð Bjarmason]

The test guarded by PERLJSON added in 75459410ed ("json_writer: new
routines to create JSON data", 2018-07-13) assumed that a JSON boolean
value like "true" or "false" would be represented as "1" or "0" in
Perl.

This behavior can't be relied upon, e.g. with JSON.pm 2.50 and
JSON::PP A JSON::PP::Boolean object will be represented as "true" or
"false". To work around this let's check if we have any refs left
after we check for hashes and arrays, assume those are JSON objects,
and coerce them to a known boolean value.

The behavior of this test still looks odd to me. Why implement our own
ad-hoc encoder just for some one-off test, as opposed to say Perl's
own Data::Dumper with Sortkeys et al? But with this change it works,
so let's leave it be.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t0019/parse_json.perl | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/t/t0019/parse_json.perl b/t/t0019/parse_json.perl
index ca4e5bfa78..fea87fb81b 100644
--- a/t/t0019/parse_json.perl
+++ b/t/t0019/parse_json.perl
@@ -34,6 +34,9 @@ sub dump_item {
     } elsif (ref($value) eq 'HASH') {
 	print "$label_in hash\n";
 	dump_hash($label_in, $value);
+    } elsif (ref $value) {
+	my $bool = $value ? 1 : 0;
+	print "$label_in $bool\n";
     } elsif (defined $value) {
 	print "$label_in $value\n";
     } else {
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v4 5/6] tests: fix version-specific portability issue in Perl JSON

[Eric Sunshine]

On Fri, Aug 24, 2018 at 11:20 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
> The test guarded by PERLJSON added in 75459410ed ("json_writer: new
> routines to create JSON data", 2018-07-13) assumed that a JSON boolean
> value like "true" or "false" would be represented as "1" or "0" in
> Perl.
>
> This behavior can't be relied upon, e.g. with JSON.pm 2.50 and
> JSON::PP A JSON::PP::Boolean object will be represented as "true" or

s/PP A/PP. A/

(Not worth a re-roll.)

> "false". To work around this let's check if we have any refs left
> after we check for hashes and arrays, assume those are JSON objects,
> and coerce them to a known boolean value.
>
> The behavior of this test still looks odd to me. Why implement our own
> ad-hoc encoder just for some one-off test, as opposed to say Perl's
> own Data::Dumper with Sortkeys et al? But with this change it works,
> so let's leave it be.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

[PATCH v4 6/6] tests: fix and add lint for non-portable grep --file

[Ævar Arnfjörð Bjarmason]

The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check
for that in the future, and fix the portability regression in
f237c8b6fe ("commit-graph: implement git-commit-graph write",
2018-04-02) that broke e.g. AIX.

1. http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl | 1 +
 t/t5318-commit-graph.sh       | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 75f38298d7..b45bdac688 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -43,6 +43,7 @@ sub err {
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
 	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
+	/\bgrep\b.*--file\b/ and err 'grep --file FILE is not portable (use grep -f FILE)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 3c1ffad491..0c500f7ca2 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -134,7 +134,7 @@ test_expect_success 'Add one more commit' '
 	git branch commits/8 &&
 	ls $objdir/pack | grep idx >existing-idx &&
 	git repack &&
-	ls $objdir/pack| grep idx | grep -v --file=existing-idx >new-idx
+	ls $objdir/pack| grep idx | grep -v -f existing-idx >new-idx
 '
 
 # Current graph structure:
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v3 4/5] tests: fix version-specific portability issue in Perl JSON

[Ævar Arnfjörð Bjarmason]

The test guarded by PERLJSON added in 75459410ed ("json_writer: new
routines to create JSON data", 2018-07-13) assumed that a JSON boolean
value like "true" or "false" would be represented as "1" or "0" in
Perl.

This behavior can't be relied upon, e.g. with JSON.pm 2.50 and
JSON::PP A JSON::PP::Boolean object will be represented as "true" or
"false". To work around this let's check if we have any refs left
after we check for hashes and arrays, assume those are JSON objects,
and coerce them to a known boolean value.

The behavior of this test still looks odd to me. Why implement our own
ad-hoc encoder just for some one-off test, as opposed to say Perl's
own Data::Dumper with Sortkeys et al? But with this change it works,
so let's leave it be.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t0019/parse_json.perl | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/t/t0019/parse_json.perl b/t/t0019/parse_json.perl
index ca4e5bfa78..fea87fb81b 100644
--- a/t/t0019/parse_json.perl
+++ b/t/t0019/parse_json.perl
@@ -34,6 +34,9 @@ sub dump_item {
     } elsif (ref($value) eq 'HASH') {
 	print "$label_in hash\n";
 	dump_hash($label_in, $value);
+    } elsif (ref $value) {
+	my $bool = $value ? 1 : 0;
+	print "$label_in $bool\n";
     } elsif (defined $value) {
 	print "$label_in $value\n";
     } else {
-- 
2.18.0.865.gffc8e1a3cd6

[PATCH v3 5/5] tests: fix and add lint for non-portable grep --file

[Ævar Arnfjörð Bjarmason]

The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check
for that in the future, and fix the portability regression in
f237c8b6fe ("commit-graph: implement git-commit-graph write",
2018-04-02) that broke e.g. AIX.

1. http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/check-non-portable-shell.pl | 1 +
 t/t5318-commit-graph.sh       | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
index 75f38298d7..b45bdac688 100755
--- a/t/check-non-portable-shell.pl
+++ b/t/check-non-portable-shell.pl
@@ -43,6 +43,7 @@ sub err {
 	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
 	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
 	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
+	/\bgrep\b.*--file\b/ and err 'grep --file FILE is not portable (use grep -f FILE)';
 	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
 	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
 		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 3c1ffad491..0c500f7ca2 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -134,7 +134,7 @@ test_expect_success 'Add one more commit' '
 	git branch commits/8 &&
 	ls $objdir/pack | grep idx >existing-idx &&
 	git repack &&
-	ls $objdir/pack| grep idx | grep -v --file=existing-idx >new-idx
+	ls $objdir/pack| grep idx | grep -v -f existing-idx >new-idx
 '
 
 # Current graph structure:
-- 
2.18.0.865.gffc8e1a3cd6

Re: [PATCH v3 5/5] tests: fix and add lint for non-portable grep --file

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check
> for that in the future, and fix the portability regression in
> f237c8b6fe ("commit-graph: implement git-commit-graph write",
> 2018-04-02) that broke e.g. AIX.
>
> 1. http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---

Thanks.

>  t/check-non-portable-shell.pl | 1 +
>  t/t5318-commit-graph.sh       | 2 +-
>  2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
> index 75f38298d7..b45bdac688 100755
> --- a/t/check-non-portable-shell.pl
> +++ b/t/check-non-portable-shell.pl
> @@ -43,6 +43,7 @@ sub err {
>  	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
>  	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
>  	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
> +	/\bgrep\b.*--file\b/ and err 'grep --file FILE is not portable (use grep -f FILE)';
>  	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
>  	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
>  		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index 3c1ffad491..0c500f7ca2 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -134,7 +134,7 @@ test_expect_success 'Add one more commit' '
>  	git branch commits/8 &&
>  	ls $objdir/pack | grep idx >existing-idx &&
>  	git repack &&
> -	ls $objdir/pack| grep idx | grep -v --file=existing-idx >new-idx
> +	ls $objdir/pack| grep idx | grep -v -f existing-idx >new-idx
>  '
>  
>  # Current graph structure:

Re: [PATCH v3 5/5] tests: fix and add lint for non-portable grep --file

[Derrick Stolee]

On 8/23/2018 4:44 PM, Junio C Hamano wrote:
> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> The --file option to grep isn't in POSIX[1], but -f is[1]. Let's check
>> for that in the future, and fix the portability regression in
>> f237c8b6fe ("commit-graph: implement git-commit-graph write",
>> 2018-04-02) that broke e.g. AIX.
>>
>> 1. http://pubs.opengroup.org/onlinepubs/009695399/utilities/grep.html
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
> Thanks.

Yes, thanks! I'll keep this in mind for the future.

>>   t/check-non-portable-shell.pl | 1 +
>>   t/t5318-commit-graph.sh       | 2 +-
>>   2 files changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
>> index 75f38298d7..b45bdac688 100755
>> --- a/t/check-non-portable-shell.pl
>> +++ b/t/check-non-portable-shell.pl
>> @@ -43,6 +43,7 @@ sub err {
>>   	/\bwc -l.*"\s*=/ and err '`"$(wc -l)"` is not portable (use test_line_count)';
>>   	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
>>   	/(?:\$\(seq|^\s*seq\b)/ and err 'seq is not portable (use test_seq)';
>> +	/\bgrep\b.*--file\b/ and err 'grep --file FILE is not portable (use grep -f FILE)';
>>   	/\bexport\s+[A-Za-z0-9_]*=/ and err '"export FOO=bar" is not portable (use FOO=bar && export FOO)';
>>   	/^\s*([A-Z0-9_]+=(\w+|(["']).*?\3)\s+)+(\w+)/ and exists($func{$4}) and
>>   		err '"FOO=bar shell_func" assignment extends beyond "shell_func"';
>> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
>> index 3c1ffad491..0c500f7ca2 100755
>> --- a/t/t5318-commit-graph.sh
>> +++ b/t/t5318-commit-graph.sh
>> @@ -134,7 +134,7 @@ test_expect_success 'Add one more commit' '
>>   	git branch commits/8 &&
>>   	ls $objdir/pack | grep idx >existing-idx &&
>>   	git repack &&
>> -	ls $objdir/pack| grep idx | grep -v --file=existing-idx >new-idx
>> +	ls $objdir/pack| grep idx | grep -v -f existing-idx >new-idx
>>   '
>>   
>>   # Current graph structure:

Re: [PATCH] tests: fix and add lint for non-portable head -c N

[Junio C Hamano]

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Junio: Even though this isn't a 2.19.0-rc0 regression I think it makes
> sense for 2.19.0. The fix is trivial, and it'll unbreak (at least some
> of) the tests on stock git on OpenBSD.

I would have been a bit more sympathetic if this were recent
regression (say, 2.18 or so), even if it is not new in this cycle,
and the patch applied to the target track cleanly (say, maint or
maint-2.17), but seeing that some of these are quite old, dating
back to 2009, I am not sure I am enthused.

The changes certainly look trivial.  Let's apply.

Thanks.

>
> OpenBSD guys: If you CC the git mailing list when you find you need to
> apply patches like these, we're happy to fix this more pro-actively. I
> just happened to be testing the upcoming 2.19.0 on OpenBSD and spotted
> this.
>
>  t/check-non-portable-shell.pl | 1 +
>  t/t5310-pack-bitmaps.sh       | 2 +-
>  t/test-lib.sh                 | 4 ++--
>  3 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
> index d5823f71d8..94a7e6165e 100755
> --- a/t/check-non-portable-shell.pl
> +++ b/t/check-non-portable-shell.pl
> @@ -35,6 +35,7 @@ sub err {
>  		chomp;
>  	}
>  
> +	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
>  	/\bsed\s+-i/ and err 'sed -i is not portable';
>  	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
>  	/^\s*declare\s+/ and err 'arrays/declare not portable';
> diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
> index 557bd0d0c0..7bff7923f2 100755
> --- a/t/t5310-pack-bitmaps.sh
> +++ b/t/t5310-pack-bitmaps.sh
> @@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
>  	git rev-list --use-bitmap-index --count --all >expect &&
>  	bitmap=$(ls .git/objects/pack/*.bitmap) &&
>  	test_when_finished "rm -f $bitmap" &&
> -	head -c 512 <$bitmap >$bitmap.tmp &&
> +	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
>  	mv -f $bitmap.tmp $bitmap &&
>  	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
>  	test_cmp expect actual &&
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 8bb0f4348e..44288cbb59 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -867,7 +867,7 @@ then
>  		# handle only executables, unless they are shell libraries that
>  		# need to be in the exec-path.
>  		test -x "$1" ||
> -		test "# " = "$(head -c 2 <"$1")" ||
> +		test "# " = "$(test_copy_bytes 2 <"$1")" ||
>  		return;
>  
>  		base=$(basename "$1")
> @@ -882,7 +882,7 @@ then
>  		# do not override scripts
>  		if test -x "$symlink_target" &&
>  		    test ! -d "$symlink_target" &&
> -		    test "#!" != "$(head -c 2 < "$symlink_target")"
> +		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
>  		then
>  			symlink_target=../valgrind.sh
>  		fi

Re: [PATCH] tests: fix and add lint for non-portable head -c N

[Ævar Arnfjörð Bjarmason]

On Thu, Aug 23 2018, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Junio: Even though this isn't a 2.19.0-rc0 regression I think it makes
>> sense for 2.19.0. The fix is trivial, and it'll unbreak (at least some
>> of) the tests on stock git on OpenBSD.
>
> I would have been a bit more sympathetic if this were recent
> regression (say, 2.18 or so), even if it is not new in this cycle,
> and the patch applied to the target track cleanly (say, maint or
> maint-2.17),

Yeah it's not a big deal if it's not in 2.19.

> but seeing that some of these are quite old, dating back to 2009, I am
> not sure I am enthused.

Note that those parts are the "while I'm at it" valgrind-specific parts,
i.e. parts of the code that likely never ran on anything except a GNU
toolchain.

> The changes certainly look trivial.  Let's apply.
>
> Thanks.
>
>>
>> OpenBSD guys: If you CC the git mailing list when you find you need to
>> apply patches like these, we're happy to fix this more pro-actively. I
>> just happened to be testing the upcoming 2.19.0 on OpenBSD and spotted
>> this.
>>
>>  t/check-non-portable-shell.pl | 1 +
>>  t/t5310-pack-bitmaps.sh       | 2 +-
>>  t/test-lib.sh                 | 4 ++--
>>  3 files changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/t/check-non-portable-shell.pl b/t/check-non-portable-shell.pl
>> index d5823f71d8..94a7e6165e 100755
>> --- a/t/check-non-portable-shell.pl
>> +++ b/t/check-non-portable-shell.pl
>> @@ -35,6 +35,7 @@ sub err {
>>  		chomp;
>>  	}
>>
>> +	/\bhead\s+-c\b/ and err 'head -c is not portable (use test_copy_bytes BYTES <file >out)';
>>  	/\bsed\s+-i/ and err 'sed -i is not portable';
>>  	/\becho\s+-[neE]/ and err 'echo with option is not portable (use printf)';
>>  	/^\s*declare\s+/ and err 'arrays/declare not portable';
>> diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
>> index 557bd0d0c0..7bff7923f2 100755
>> --- a/t/t5310-pack-bitmaps.sh
>> +++ b/t/t5310-pack-bitmaps.sh
>> @@ -335,7 +335,7 @@ test_expect_success 'truncated bitmap fails gracefully' '
>>  	git rev-list --use-bitmap-index --count --all >expect &&
>>  	bitmap=$(ls .git/objects/pack/*.bitmap) &&
>>  	test_when_finished "rm -f $bitmap" &&
>> -	head -c 512 <$bitmap >$bitmap.tmp &&
>> +	test_copy_bytes 512 <$bitmap >$bitmap.tmp &&
>>  	mv -f $bitmap.tmp $bitmap &&
>>  	git rev-list --use-bitmap-index --count --all >actual 2>stderr &&
>>  	test_cmp expect actual &&
>> diff --git a/t/test-lib.sh b/t/test-lib.sh
>> index 8bb0f4348e..44288cbb59 100644
>> --- a/t/test-lib.sh
>> +++ b/t/test-lib.sh
>> @@ -867,7 +867,7 @@ then
>>  		# handle only executables, unless they are shell libraries that
>>  		# need to be in the exec-path.
>>  		test -x "$1" ||
>> -		test "# " = "$(head -c 2 <"$1")" ||
>> +		test "# " = "$(test_copy_bytes 2 <"$1")" ||
>>  		return;
>>
>>  		base=$(basename "$1")
>> @@ -882,7 +882,7 @@ then
>>  		# do not override scripts
>>  		if test -x "$symlink_target" &&
>>  		    test ! -d "$symlink_target" &&
>> -		    test "#!" != "$(head -c 2 < "$symlink_target")"
>> +		    test "#!" != "$(test_copy_bytes 2 <"$symlink_target")"
>>  		then
>>  			symlink_target=../valgrind.sh
>>  		fi