[PATCH] convert: fail gracefully upon missing clean cmd on required filter

[PATCH] convert: fail gracefully upon missing clean cmd on required filter

[Matheus Tavares]

The gitattributes documentation mentions that either the clean cmd or
the smudge cmd can be left unspecified in a filter definition. However,
when the filter is marked as 'required', the absence of any one of these
two should be treated as an error. Git already fails under these
circumstances, but not always in a pleasant way: omitting a clean cmd in
a required filter triggers an assertion error which leaves the user with
a quite verbose message:

git: convert.c:1462: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.

This assertion and the one above it are not really necessary, as the
apply_filter() call bellow them already performs the same checks. And
when these conditions are not met, the function returns 0, making the
caller die() with a much nicer message. (Also note that die()-ing here
is the right behavior as `would_convert_to_git_filter_fd() == true` is a
precondition to use convert_to_git_filter_fd(), and the former is only
true when the filter is required.) So remove both assertions and add two
regression tests to make sure that git fails nicely when either the
smudge or clean command is missing on a required filter.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 convert.c             |  3 ---
 t/t0021-conversion.sh | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 3 deletions(-)

diff --git a/convert.c b/convert.c
index ee360c2f07..48fbdac676 100644
--- a/convert.c
+++ b/convert.c
@@ -1455,9 +1455,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,
 	struct conv_attrs ca;
 	convert_attrs(istate, &ca, path);
 
-	assert(ca.drv);
-	assert(ca.drv->clean || ca.drv->process);
-
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
 		die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
 
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index e828ee964c..4f8415d419 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '
 	test_must_fail git add test.fc
 '
 
+test_expect_success 'required filter with absent clean field' '
+	test_config filter.absentclean.smudge cat &&
+	test_config filter.absentclean.required true &&
+
+	echo "*.ac filter=absentclean" >.gitattributes &&
+
+	echo test >test.ac &&
+	test_must_fail git add test.ac 2>stderr &&
+	test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr
+'
+
+test_expect_success 'required filter with absent smudge field' '
+	test_config filter.absentsmudge.clean cat &&
+	test_config filter.absentsmudge.required true &&
+
+	echo "*.as filter=absentsmudge" >.gitattributes &&
+
+	echo test >test.as &&
+	git add test.as &&
+	rm -f test.as &&
+	test_must_fail git checkout -- test.as 2>stderr &&
+	test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr
+'
+
 test_expect_success 'filtering large input to small output should use little memory' '
 	test_config filter.devnull.clean "cat >/dev/null" &&
 	test_config filter.devnull.required true &&
-- 
2.30.1

Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter

[Junio C Hamano]

Matheus Tavares <matheus.bernardino@usp.br> writes:

> The gitattributes documentation mentions that either the clean cmd or
> the smudge cmd can be left unspecified in a filter definition. However,
> when the filter is marked as 'required', the absence of any one of these
> two should be treated as an error. Git already fails under these
> circumstances, but not always in a pleasant way: omitting a clean cmd in
> a required filter triggers an assertion error which leaves the user with
> a quite verbose message:
>
> git: convert.c:1462: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.
>
> This assertion and the one above it are not really necessary, as the
> apply_filter() call bellow them already performs the same checks. And
> when these conditions are not met, the function returns 0, making the
> caller die() with a much nicer message. (Also note that die()-ing here
> is the right behavior as `would_convert_to_git_filter_fd() == true` is a
> precondition to use convert_to_git_filter_fd(), and the former is only
> true when the filter is required.) So remove both assertions and add two
> regression tests to make sure that git fails nicely when either the
> smudge or clean command is missing on a required filter.

Makes sense.

Will queue, thanks.

Re: [PATCH] convert: fail gracefully upon missing clean cmd on required filter

[Martin Ågren]

On Thu, 25 Feb 2021 at 19:18, Matheus Tavares <matheus.bernardino@usp.br> wrote:
> This assertion and the one above it are not really necessary, as the
> apply_filter() call bellow them already performs the same checks. And

s/bellow/below/

> when these conditions are not met, the function returns 0, making the
> caller die() with a much nicer message. (Also note that die()-ing here

Makes sense. I noticed one thing:

> -       assert(ca.drv);

If ca.drv is NULL ....

> -       assert(ca.drv->clean || ca.drv->process);
> -
>         if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))

... the return value will be 0, so this will trigger ...

>                 die(_("%s: clean filter '%s' failed"), path, ca.drv->name);

... and we'll dereference NULL to grab the name.

It seems like you could leave that first assertion and your new tests
would still pass. Hitting an assertion is arguably better than wandering
off into undefined behavior. (What will probably happen is ca.drv->name
will effectively also be NULL, because it's at the top of the struct.
Some implementations will format this as "(null)", others will crash.)

As you note, and my reading agrees, you can't really have ca.drv be NULL
here. So this is like, if and when we grow a bug somewhere and actually
do have NULL, maybe we would rather hit that "assert" than go
dereferencing NULL.

Martin

[PATCH v2] convert: fail gracefully upon missing clean cmd on required filter

[Matheus Tavares]

The gitattributes documentation mentions that either the clean cmd or
the smudge cmd can be left unspecified in a filter definition. However,
when the filter is marked as 'required', the absence of any one of these
two should be treated as an error. Git already fails under these
circumstances, but not always in a pleasant way: omitting a clean cmd in
a required filter triggers an assertion error which leaves the user with
a quite verbose message:

git: convert.c:1459: convert_to_git_filter_fd: Assertion "ca.drv->clean || ca.drv->process" failed.

This assertion is not really necessary, as the apply_filter() call below
it already performs the same check. And when this condition is not met,
the function returns 0, making the caller die() with a much nicer
message. (Also note that die()-ing here is the right behavior as
`would_convert_to_git_filter_fd() == true` is a precondition to use
convert_to_git_filter_fd(), and the former is only true when the filter
is required.) So remove the assertion and add two regression tests to
make sure that git fails nicely when either the smudge or clean command
is missing on a required filter.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---

Changes since v1: didn't remove the first assertion and adjusted the
commit message. Thanks for catching that, Martin.

 convert.c             |  1 -
 t/t0021-conversion.sh | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/convert.c b/convert.c
index ee360c2f07..2c2c2adf61 100644
--- a/convert.c
+++ b/convert.c
@@ -1456,7 +1456,6 @@ void convert_to_git_filter_fd(const struct index_state *istate,
 	convert_attrs(istate, &ca, path);
 
 	assert(ca.drv);
-	assert(ca.drv->clean || ca.drv->process);
 
 	if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
 		die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index e828ee964c..4f8415d419 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -257,6 +257,30 @@ test_expect_success 'required filter clean failure' '
 	test_must_fail git add test.fc
 '
 
+test_expect_success 'required filter with absent clean field' '
+	test_config filter.absentclean.smudge cat &&
+	test_config filter.absentclean.required true &&
+
+	echo "*.ac filter=absentclean" >.gitattributes &&
+
+	echo test >test.ac &&
+	test_must_fail git add test.ac 2>stderr &&
+	test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr
+'
+
+test_expect_success 'required filter with absent smudge field' '
+	test_config filter.absentsmudge.clean cat &&
+	test_config filter.absentsmudge.required true &&
+
+	echo "*.as filter=absentsmudge" >.gitattributes &&
+
+	echo test >test.as &&
+	git add test.as &&
+	rm -f test.as &&
+	test_must_fail git checkout -- test.as 2>stderr &&
+	test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr
+'
+
 test_expect_success 'filtering large input to small output should use little memory' '
 	test_config filter.devnull.clean "cat >/dev/null" &&
 	test_config filter.devnull.required true &&
-- 
2.30.1

Re: [PATCH v2] convert: fail gracefully upon missing clean cmd on required filter

[Martin Ågren]

On Fri, 26 Feb 2021 at 17:24, Matheus Tavares <matheus.bernardino@usp.br> wrote:
> This assertion is not really necessary, as the apply_filter() call below
> it already performs the same check. And when this condition is not met,

> -       assert(ca.drv->clean || ca.drv->process);

FWIW, this LGTM.

> +       test_i18ngrep "fatal: test.ac: clean filter .absentclean. failed" stderr

> +       test_i18ngrep "fatal: test.as: smudge filter absentsmudge failed" stderr

(Funny how these two error messages differ in quoting. That's obviously
not anything your patch needs to fix.)

Martin