git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH] archive: add test testing MIME for created archive through compression filter
@ 2021-10-11 11:27 Bagas Sanjaya
  2021-10-11 15:48 ` Junio C Hamano
  2021-10-11 15:53 ` Junio C Hamano
  0 siblings, 2 replies; 6+ messages in thread
From: Bagas Sanjaya @ 2021-10-11 11:27 UTC (permalink / raw)
  To: git; +Cc: René Scharfe, Jeff King, Bagas Sanjaya

The common use of `tar.<format>.command` config is to specify
compression program filter for creating compressed tar archive.

Add a test that tests MIME type of archives created through the filter.
The generated archives must not be `application/x-tar` (POSIX tar
archive) type, so that these can be decompressed with the corresponding
decompression program (such as gunzip, bunzip2, and unxz).

Signed-off-by: Bagas Sanjaya <bagasdotme@gmail.com>
---
 t/t5000-tar-tree.sh | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/t/t5000-tar-tree.sh b/t/t5000-tar-tree.sh
index 2c88d1c159..04cdad5bff 100755
--- a/t/t5000-tar-tree.sh
+++ b/t/t5000-tar-tree.sh
@@ -38,6 +38,8 @@ test_lazy_prereq TAR_NEEDS_PAX_FALLBACK '
 '
 
 test_lazy_prereq GZIP 'gzip --version'
+test_lazy_prereq BZIP2 'bzip2 --version'
+test_lazy_prereq XZ 'xz --version'
 
 get_pax_header() {
 	file=$1
@@ -374,6 +376,21 @@ test_expect_success GZIP 'remote tar.gz can be disabled' '
 		>remote.tar.gz
 '
 
+test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '
+	git config tar.tar.bz2.command "bzip2 -c" &&
+	git config tar.tar.xz.command "xz -c" &&
+	git archive --output HEAD.tar.gz --prefix=src/ HEAD &&
+	git archive --output HEAD.tar.bz2 --prefix=src/ HEAD &&
+	git archive --output HEAD.tar.xz --prefix=src/ HEAD &&
+	cat >expect <<EOF &&
+HEAD.tar.bz2: application/x-bzip2; charset=binary
+HEAD.tar.gz:  application/gzip; charset=binary
+HEAD.tar.xz:  application/x-xz; charset=binary
+EOF
+	file -i HEAD.tar.bz2 HEAD.tar.gz HEAD.tar.xz 2>/dev/null >actual &&
+	test_cmp expect actual
+'
+
 test_expect_success 'archive and :(glob)' '
 	git archive -v HEAD -- ":(glob)**/sh" >/dev/null 2>actual &&
 	cat >expect <<EOF &&

base-commit: 106298f7f9cca4158a980de149ef217751e1f943
-- 
An old man doll... just what I always wanted! - Clara


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] archive: add test testing MIME for created archive through compression filter
  2021-10-11 11:27 [PATCH] archive: add test testing MIME for created archive through compression filter Bagas Sanjaya
@ 2021-10-11 15:48 ` Junio C Hamano
  2021-10-11 15:53 ` Junio C Hamano
  1 sibling, 0 replies; 6+ messages in thread
From: Junio C Hamano @ 2021-10-11 15:48 UTC (permalink / raw)
  To: Bagas Sanjaya; +Cc: git, René Scharfe, Jeff King

Bagas Sanjaya <bagasdotme@gmail.com> writes:

> +	cat >expect <<EOF &&
> +HEAD.tar.bz2: application/x-bzip2; charset=binary
> +HEAD.tar.gz:  application/gzip; charset=binary
> +HEAD.tar.xz:  application/x-xz; charset=binary
> +EOF

Use <<-EOF and you can align here-document to the script.

> +	file -i HEAD.tar.bz2 HEAD.tar.gz HEAD.tar.xz 2>/dev/null >actual &&

This looks super un-portable.

  https://pubs.opengroup.org/onlinepubs/9699919799/utilities/file.html



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] archive: add test testing MIME for created archive through compression filter
  2021-10-11 11:27 [PATCH] archive: add test testing MIME for created archive through compression filter Bagas Sanjaya
  2021-10-11 15:48 ` Junio C Hamano
@ 2021-10-11 15:53 ` Junio C Hamano
  2021-10-11 16:07   ` Jeff King
  2021-10-12  3:14   ` Bagas Sanjaya
  1 sibling, 2 replies; 6+ messages in thread
From: Junio C Hamano @ 2021-10-11 15:53 UTC (permalink / raw)
  To: Bagas Sanjaya; +Cc: git, René Scharfe, Jeff King

Bagas Sanjaya <bagasdotme@gmail.com> writes:

> +test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '
> +	git config tar.tar.bz2.command "bzip2 -c" &&
> +	git config tar.tar.xz.command "xz -c" &&
> +	git archive --output HEAD.tar.gz --prefix=src/ HEAD &&

I think a lot more portable and robust way to test the feature is to
configure git config tar.tar.test.command (or come up with a name for
a suffix to be used in the test), point it at a script created in this
test script and run git archive with output filename that would trigger
the command.  Then, arrange the test to notice if the "test" script was
called with expected command line arguments and standard input.

That way, you do not need to rely on prereqs and you do not have to
resort to un-portable use of the "file" command.  After all, you are
*not* testing if "bzip2 -c" the user happens to have on their $PATH
produces output their "find" recognises as bzip2 compressed.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] archive: add test testing MIME for created archive through compression filter
  2021-10-11 15:53 ` Junio C Hamano
@ 2021-10-11 16:07   ` Jeff King
  2021-10-12  3:14   ` Bagas Sanjaya
  1 sibling, 0 replies; 6+ messages in thread
From: Jeff King @ 2021-10-11 16:07 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Bagas Sanjaya, git, René Scharfe

On Mon, Oct 11, 2021 at 08:53:11AM -0700, Junio C Hamano wrote:

> Bagas Sanjaya <bagasdotme@gmail.com> writes:
> 
> > +test_expect_success GZIP,BZIP2,XZ 'git archive with gzip, bzip2, and xz filters creates compressed tar archive with proper MIME type' '
> > +	git config tar.tar.bz2.command "bzip2 -c" &&
> > +	git config tar.tar.xz.command "xz -c" &&
> > +	git archive --output HEAD.tar.gz --prefix=src/ HEAD &&
> 
> I think a lot more portable and robust way to test the feature is to
> configure git config tar.tar.test.command (or come up with a name for
> a suffix to be used in the test), point it at a script created in this
> test script and run git archive with output filename that would trigger
> the command.  Then, arrange the test to notice if the "test" script was
> called with expected command line arguments and standard input.
> 
> That way, you do not need to rely on prereqs and you do not have to
> resort to un-portable use of the "file" command.  After all, you are
> *not* testing if "bzip2 -c" the user happens to have on their $PATH
> produces output their "find" recognises as bzip2 compressed.

That would work. Or perhaps using something portable like "tr" to
do a nonsense conversion, and verifying that we can get convert back to
the original. Like say, the tests added to t5000 by 767cf4579f (archive:
implement configurable tar filters, 2011-06-21).

I don't think the newly proposed test is adding anything beyond that
(except checking the system "bzip2" command, but as you say, that is not
useful to us, nor portable).

-Peff

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] archive: add test testing MIME for created archive through compression filter
  2021-10-11 15:53 ` Junio C Hamano
  2021-10-11 16:07   ` Jeff King
@ 2021-10-12  3:14   ` Bagas Sanjaya
  2021-10-12 19:15     ` Junio C Hamano
  1 sibling, 1 reply; 6+ messages in thread
From: Bagas Sanjaya @ 2021-10-12  3:14 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, René Scharfe, Jeff King

On 11/10/21 22.53, Junio C Hamano wrote:
> I think a lot more portable and robust way to test the feature is to
> configure git config tar.tar.test.command (or come up with a name for
> a suffix to be used in the test), point it at a script created in this
> test script and run git archive with output filename that would trigger
> the command.  Then, arrange the test to notice if the "test" script was
> called with expected command line arguments and standard input.
> 
> That way, you do not need to rely on prereqs and you do not have to
> resort to un-portable use of the "file" command.  After all, you are
> *not* testing if "bzip2 -c" the user happens to have on their $PATH
> produces output their "find" recognises as bzip2 compressed.
> 

The intent of this test is to ensure `git archive -o 
something.tar.<format>` produces proper compressed tar archive that can 
be decompressed with the corresponding decompression tool (such as 
gunzip for gz files), and not just with `tar xvf`.

-- 
An old man doll... just what I always wanted! - Clara

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] archive: add test testing MIME for created archive through compression filter
  2021-10-12  3:14   ` Bagas Sanjaya
@ 2021-10-12 19:15     ` Junio C Hamano
  0 siblings, 0 replies; 6+ messages in thread
From: Junio C Hamano @ 2021-10-12 19:15 UTC (permalink / raw)
  To: Bagas Sanjaya; +Cc: git, René Scharfe, Jeff King

Bagas Sanjaya <bagasdotme@gmail.com> writes:

> On 11/10/21 22.53, Junio C Hamano wrote:
>> I think a lot more portable and robust way to test the feature is to
>> configure git config tar.tar.test.command (or come up with a name for
>> a suffix to be used in the test), point it at a script created in this
>> test script and run git archive with output filename that would trigger
>> the command.  Then, arrange the test to notice if the "test" script was
>> called with expected command line arguments and standard input.
>> That way, you do not need to rely on prereqs and you do not have to
>> resort to un-portable use of the "file" command.  After all, you are
>> *not* testing if "bzip2 -c" the user happens to have on their $PATH
>> produces output their "find" recognises as bzip2 compressed.
>> 
>
> The intent of this test is to ensure `git archive -o
> something.tar.<format>` produces proper compressed tar archive that
> can be decompressed with the corresponding decompression tool (such as 
> gunzip for gz files), and not just with `tar xvf`.

Yes, that is why relying on "file -i" is not the right thing to do.

We'd want to use the matching decompressor, or better yet, trust
that the compressor-decompressor pair the system offers do work
correctly (after all, it's not our test suite's job to debug the
system software the tester uses) and validate that

 (1) the specified compressor is spawned with the right parameters
     taken from the configuration file, and that

 (2) the compressor is fed the correct input stream.

Using a script prepared by the test itself as the "compressor" would
be one good way to validate (1); it can record how it was invoked,
so that the test can check its correctness.  Using a reversible
transformation in the script as a fake compressor and comparing the
reversed output with the output without any compression filter, may
be one way to validate (2), like Peff mentioned.

But asking "file -i" validates neither.

The code that reads the configuration and spawns the compressor with
specified parameters may be broken and may not pass the right
parameters, or the archive stream may not be passed to the
compressor intact.  But the compressor may still produce a header
block that is looking correct enough for the "file" command to say
"ah, that uses X compressor", and the tests in the patch under
discussion does not even see if the payload was passed correctly
without corruption to the compressor at all.

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-10-12 19:15 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-11 11:27 [PATCH] archive: add test testing MIME for created archive through compression filter Bagas Sanjaya
2021-10-11 15:48 ` Junio C Hamano
2021-10-11 15:53 ` Junio C Hamano
2021-10-11 16:07   ` Jeff King
2021-10-12  3:14   ` Bagas Sanjaya
2021-10-12 19:15     ` Junio C Hamano

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).