git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Eric Sunshine <sunshine@sunshineco.com>
To: git@vger.kernel.org
Cc: Jonathan Nieder <jrnieder@gmail.com>, Jeff King <peff@peff.net>,
	Eric Sunshine <sunshine@sunshineco.com>
Subject: [PATCH 4/5] chainlint: recognize multi-line quoted strings more robustly
Date: Tue,  7 Aug 2018 04:21:34 -0400	[thread overview]
Message-ID: <20180807082135.60913-5-sunshine@sunshineco.com> (raw)
In-Reply-To: <20180807082135.60913-1-sunshine@sunshineco.com>

chainlint.sed recognizes multi-line quoted strings within subshells:

    echo "abc
        def" >out &&

so it can avoid incorrectly classifying lines internal to the string as
breaking the &&-chain. To identify the first line of a multi-line
string, it checks if the line contains a single quote. However, this is
fragile and can be easily fooled by a line containing multiple strings:

    echo "xyz" "abc
        def" >out &&

Make detection more robust by checking for an odd number of quotes
rather than only a single one.

(Escaped quotes are not handled, but support may be added later.)

The original multi-line string recognizer rather cavalierly threw away
all but the final quote, whereas the new one is careful to retain all
quotes, so the "expected" output of a couple existing chainlint tests is
updated to account for this new behavior.

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.sed                               | 32 +++++++++++++------
 t/chainlint/here-doc-multi-line-string.expect |  2 +-
 t/chainlint/multi-line-string.expect          | 10 ++++--
 t/chainlint/multi-line-string.test            | 12 +++++++
 4 files changed, 43 insertions(+), 13 deletions(-)

diff --git a/t/chainlint.sed b/t/chainlint.sed
index 6c891bf383..338163c2f5 100644
--- a/t/chainlint.sed
+++ b/t/chainlint.sed
@@ -151,10 +151,10 @@ s/.*\n//
 :slurp
 # incomplete line "...\"
 /\\$/bincomplete
-# multi-line quoted string "...\n..."
-/^[^"]*"[^"]*$/bdqstring
-# multi-line quoted string '...\n...' (but not contraction in string "it's so")
-/^[^']*'[^']*$/{
+# multi-line quoted string "...\n..."?
+/"/bdqstring
+# multi-line quoted string '...\n...'? (but not contraction in string "it's")
+/'/{
 	/"[^'"]*'[^'"]*"/!bsqstring
 }
 :folded
@@ -250,20 +250,32 @@ N
 s/\\\n//
 bslurp
 
-# found multi-line double-quoted string "...\n..." -- slurp until end of string
+# check for multi-line double-quoted string "...\n..." -- fold to one line
 :dqstring
-s/"//g
+# remove all quote pairs
+s/"\([^"]*\)"/@!\1@!/g
+# done if no dangling quote
+/"/!bdqdone
+# otherwise, slurp next line and try again
 N
 s/\n//
-/"/!bdqstring
+bdqstring
+:dqdone
+s/@!/"/g
 bfolded
 
-# found multi-line single-quoted string '...\n...' -- slurp until end of string
+# check for multi-line single-quoted string '...\n...' -- fold to one line
 :sqstring
-s/'//g
+# remove all quote pairs
+s/'\([^']*\)'/@!\1@!/g
+# done if no dangling quote
+/'/!bsqdone
+# otherwise, slurp next line and try again
 N
 s/\n//
-/'/!bsqstring
+bsqstring
+:sqdone
+s/@!/'/g
 bfolded
 
 # found here-doc -- swallow it to avoid false hits within its body (but keep
diff --git a/t/chainlint/here-doc-multi-line-string.expect b/t/chainlint/here-doc-multi-line-string.expect
index 1e5b724b9d..32038a070c 100644
--- a/t/chainlint/here-doc-multi-line-string.expect
+++ b/t/chainlint/here-doc-multi-line-string.expect
@@ -1,4 +1,4 @@
 (
-?!AMP?!	cat && echo multi-line	string"
+?!AMP?!	cat && echo "multi-line	string"
 	bap
 >)
diff --git a/t/chainlint/multi-line-string.expect b/t/chainlint/multi-line-string.expect
index 8334c4cc8e..170cb59993 100644
--- a/t/chainlint/multi-line-string.expect
+++ b/t/chainlint/multi-line-string.expect
@@ -1,9 +1,15 @@
 (
-	x=line 1		line 2		line 3" &&
-?!AMP?!	y=line 1		line2'
+	x="line 1		line 2		line 3" &&
+?!AMP?!	y='line 1		line2'
 	foobar
 >) &&
 (
 	echo "there's nothing to see here" &&
 	exit
+>) &&
+(
+	echo "xyz" "abc		def		ghi" &&
+	echo 'xyz' 'abc		def		ghi' &&
+	echo 'xyz' "abc		def		ghi" &&
+	barfoo
 >)
diff --git a/t/chainlint/multi-line-string.test b/t/chainlint/multi-line-string.test
index 14cb44d51c..287ab89705 100644
--- a/t/chainlint/multi-line-string.test
+++ b/t/chainlint/multi-line-string.test
@@ -12,4 +12,16 @@
 # LINT: starting multi-line single-quoted string
 	echo "there's nothing to see here" &&
 	exit
+) &&
+(
+	echo "xyz" "abc
+		def
+		ghi" &&
+	echo 'xyz' 'abc
+		def
+		ghi' &&
+	echo 'xyz' "abc
+		def
+		ghi" &&
+	barfoo
 )
-- 
2.18.0.758.g1932418f46


  parent reply	other threads:[~2018-08-07  8:22 UTC|newest]

Thread overview: 123+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-26  7:29 [PATCH 00/29] t: detect and fix broken &&-chains in subshells Eric Sunshine
2018-06-26  7:29 ` [PATCH 01/29] t7508: use test_when_finished() instead of managing exit code manually Eric Sunshine
2018-06-26  7:29 ` [PATCH 02/29] t0001: use "{...}" block around "||" expression rather than subshell Eric Sunshine
2018-06-26  7:29 ` [PATCH 03/29] t1300: use sane_unset() to avoid breaking &&-chain Eric Sunshine
2018-06-26  7:29 ` [PATCH 04/29] t3303: use standard here-doc tag "EOF" to avoid fooling --chain-lint Eric Sunshine
2018-06-26  7:29 ` [PATCH 05/29] t5505: modernize and simplify hard-to-digest test Eric Sunshine
2018-06-26  7:29 ` [PATCH 06/29] t6036: fix broken "merge fails but has appropriate contents" tests Eric Sunshine
2018-06-26  8:44   ` Elijah Newren
2018-06-26  7:29 ` [PATCH 07/29] t7201: drop pointless "exit 0" at end of subshell Eric Sunshine
2018-06-26  7:29 ` [PATCH 08/29] t7400: fix broken "submodule add/reconfigure --force" test Eric Sunshine
2018-06-27 18:04   ` Stefan Beller
2018-06-26  7:29 ` [PATCH 09/29] t7810: use test_expect_code() instead of hand-rolled comparison Eric Sunshine
2018-06-26  7:29 ` [PATCH 10/29] t9001: fix broken "invoke hook" test Eric Sunshine
2018-06-26 17:07   ` Jonathan Tan
2018-06-26  7:29 ` [PATCH 11/29] t9104: use "{...}" block around "||" expression rather than subshell Eric Sunshine
2018-06-26  7:29 ` [PATCH 12/29] t9401: drop unnecessary nested subshell Eric Sunshine
2018-06-26  7:29 ` [PATCH 13/29] t/lib-submodule-update: fix broken "replace submodule must-fail" test Eric Sunshine
2018-06-27 18:30   ` [PATCH] t/lib-submodule-update: fix absorbing test Stefan Beller
2018-06-27 18:38     ` Eric Sunshine
2018-06-26  7:29 ` [PATCH 14/29] t: drop subshell with missing &&-chain in favor of simpler construct Eric Sunshine
2018-06-26 19:31   ` Junio C Hamano
2018-06-26 20:06     ` Eric Sunshine
2018-06-26  7:29 ` [PATCH 15/29] t: drop unnecessary terminating semicolons in subshell Eric Sunshine
2018-06-26  7:29 ` [PATCH 16/29] t: use test_might_fail() instead of manipulating exit code manually Eric Sunshine
2018-06-26  7:29 ` [PATCH 17/29] t: use test_must_fail() instead of checking " Eric Sunshine
2018-06-26  7:59   ` Luke Diamand
2018-06-26  8:58   ` Elijah Newren
2018-06-26  9:21     ` Eric Sunshine
2018-06-26 18:05       ` Johannes Sixt
2018-06-26 18:14         ` Eric Sunshine
2018-06-26 21:00           ` Johannes Sixt
2018-06-26  7:29 ` [PATCH 18/29] t0000-t0999: fix broken &&-chains in subshells Eric Sunshine
2018-06-26  7:29 ` [PATCH 19/29] t1000-t1999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 20/29] t2000-t2999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 21/29] t3000-t3999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 22/29] t3030: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 23/29] t4000-t4999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 24/29] t5000-t5999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 25/29] t6000-t6999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 26/29] t7000-t7999: " Eric Sunshine
2018-06-26  7:29 ` [PATCH 27/29] t9000-t9999: " Eric Sunshine
2018-06-26  7:30 ` [PATCH 28/29] t9119: " Eric Sunshine
2018-06-26  7:30 ` [PATCH 29/29] t/test-lib: teach --chain-lint to detect " Eric Sunshine
2018-06-26 19:15   ` Junio C Hamano
2018-06-26 19:52     ` Eric Sunshine
2018-06-26 20:17       ` Jeff King
2018-06-26 20:22         ` Jeff King
2018-06-26 20:59           ` Eric Sunshine
2018-06-26 21:33           ` Elijah Newren
2018-06-26 21:42             ` Eric Sunshine
2018-06-26 20:46         ` Eric Sunshine
2018-06-26 21:01           ` Jeff King
2018-06-26 21:13             ` Eric Sunshine
2018-06-28 14:35               ` Jeff King
2018-06-27  2:15             ` Elijah Newren
2018-06-27  6:27               ` Johannes Sixt
2018-06-27  6:48                 ` Eric Sunshine
2018-06-28 14:37               ` Jeff King
2018-06-26 21:09         ` Junio C Hamano
2018-06-26  9:20 ` [PATCH 00/29] t: detect and fix " Elijah Newren
2018-06-26  9:31   ` Eric Sunshine
2018-06-26 15:34     ` Elijah Newren
2018-06-26 19:38 ` Junio C Hamano
2018-06-26 21:25   ` Eric Sunshine
2018-06-26 22:31     ` Junio C Hamano
2018-06-27  0:22       ` Jonathan Nieder
2018-07-11  6:46 ` [PATCH v2 00/10] detect " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 01/10] t/test-lib: teach --chain-lint to " Eric Sunshine
2018-07-11 21:37     ` Junio C Hamano
2018-07-12 10:50       ` Eric Sunshine
2018-07-12 16:56         ` Jeff King
2018-07-12 19:32           ` Eric Sunshine
2018-07-12 19:54             ` Junio C Hamano
2018-07-30 18:13     ` Jonathan Nieder
2018-07-30 19:06       ` [PATCH 0/2] subtree: fix &&-chain and simplify tests (Re: [PATCH v2 01/10] t/test-lib: teach --chain-lint to detect broken &&-chains in subshells) Jonathan Nieder
2018-07-30 19:07         ` [PATCH 1/2] subtree test: add missing && to &&-chain Jonathan Nieder
2018-07-30 19:07         ` [PATCH 2/2] subtree test: simplify preparation of expected results Jonathan Nieder
2018-07-30 20:25       ` [PATCH v2 01/10] t/test-lib: teach --chain-lint to detect broken &&-chains in subshells Eric Sunshine
2018-07-30 20:59         ` Jonathan Nieder
2018-07-30 21:38           ` Eric Sunshine
2018-07-31 12:50             ` Jeff King
2018-07-31 18:55               ` Eric Sunshine
2018-07-31 19:08                 ` Jeff King
2018-08-23 18:02     ` Ævar Arnfjörð Bjarmason
2018-08-23 18:27       ` Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 02/10] t/Makefile: add machinery to check correctness of chainlint.sed Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 03/10] t/chainlint: add chainlint "basic" test cases Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 04/10] t/chainlint: add chainlint "whitespace" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 05/10] t/chainlint: add chainlint "one-liner" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 06/10] t/chainlint: add chainlint "nested subshell" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 07/10] t/chainlint: add chainlint "loop" and "conditional" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 08/10] t/chainlint: add chainlint "cuddled" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 09/10] t/chainlint: add chainlint "complex" " Eric Sunshine
2018-07-11  6:46   ` [PATCH v2 10/10] t/chainlint: add chainlint "specialized" " Eric Sunshine
2018-08-07  8:21   ` [PATCH 0/5] chainlint: improve robustness against "unusual" shell coding Eric Sunshine
2018-08-07  8:21     ` [PATCH 1/5] chainlint: match arbitrary here-docs tags rather than hard-coded names Eric Sunshine
2018-08-08 22:50       ` Jeff King
2018-08-09  5:58         ` Eric Sunshine
2018-08-09 14:26           ` Jeff King
2018-08-07  8:21     ` [PATCH 2/5] chainlint: recognize multi-line $(...) when command cuddled with "$(" Eric Sunshine
2018-08-07  8:21     ` [PATCH 3/5] chainlint: let here-doc and multi-line string commence on same line Eric Sunshine
2018-08-07  8:21     ` Eric Sunshine [this message]
2018-08-07  8:21     ` [PATCH 5/5] chainlint: add test of pathological case which triggered false positive Eric Sunshine
2018-08-08 22:53     ` [PATCH 0/5] chainlint: improve robustness against "unusual" shell coding Jeff King
2018-08-09  0:44       ` Junio C Hamano
2018-08-13  8:47     ` [PATCH v2 0/6] " Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 1/6] chainlint: match arbitrary here-docs tags rather than hard-coded names Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 2/6] chainlint: match 'quoted' here-doc tags Eric Sunshine
2018-08-13 19:27         ` Junio C Hamano
2018-08-13 20:12           ` Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 3/6] chainlint: recognize multi-line $(...) when command cuddled with "$(" Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 4/6] chainlint: let here-doc and multi-line string commence on same line Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 5/6] chainlint: recognize multi-line quoted strings more robustly Eric Sunshine
2018-08-13  8:47       ` [PATCH v2 6/6] chainlint: add test of pathological case which triggered false positive Eric Sunshine
2018-08-15 18:45       ` [PATCH v3 0/6] chainlint: improve robustness against "unusual" shell coding Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 1/6] chainlint: match arbitrary here-docs tags rather than hard-coded names Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 2/6] chainlint: match quoted here-doc tags Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 3/6] chainlint: recognize multi-line $(...) when command cuddled with "$(" Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 4/6] chainlint: let here-doc and multi-line string commence on same line Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 5/6] chainlint: recognize multi-line quoted strings more robustly Eric Sunshine
2018-08-15 18:45         ` [PATCH v3 6/6] chainlint: add test of pathological case which triggered false positive Eric Sunshine
2018-08-29  9:45         ` [PATCH] chainlint: match "quoted" here-doc tags Eric Sunshine
2018-08-29 17:57           ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180807082135.60913-5-sunshine@sunshineco.com \
    --to=sunshine@sunshineco.com \
    --cc=git@vger.kernel.org \
    --cc=jrnieder@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).