From: Jakub Narebski <jnareb@gmail.com>
To: Derrick Stolee <dstolee@microsoft.com>
Cc: git@vger.kernel.org, "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Martin Ågren" <martin.agren@gmail.com>,
"Jeff King" <peff@peff.net>,
"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: Re: [PATCH v2 03/12] commit-graph: test that 'verify' finds corruption
Date: Mon, 21 May 2018 20:53:29 +0200 [thread overview]
Message-ID: <86wovwdemu.fsf@gmail.com> (raw)
In-Reply-To: <20180511211504.79877-4-dstolee@microsoft.com> (Derrick Stolee's message of "Fri, 11 May 2018 21:15:17 +0000")
Derrick Stolee <dstolee@microsoft.com> writes:
> Add test cases to t5318-commit-graph.sh that corrupt the commit-graph
> file and check that the 'git commit-graph verify' command fails. These
> tests verify the header and chunk information is checked carefully.
>
> Helped-by: Martin Ågren <martin.agren@gmail.com>
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
> t/t5318-commit-graph.sh | 53 +++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 53 insertions(+)
>
> diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
> index 6ca451dfd2..0cb88232fa 100755
> --- a/t/t5318-commit-graph.sh
> +++ b/t/t5318-commit-graph.sh
> @@ -240,4 +240,57 @@ test_expect_success 'git commit-graph verify' '
> git commit-graph verify >output
> '
>
> +# usage: corrupt_data <file> <pos> [<data>]
> +corrupt_data() {
> + file=$1
> + pos=$2
> + data="${3:-\0}"
> + printf "$data" | dd of="$file" bs=1 seek="$pos" conv=notrunc
> +}
First, if we do this that way (and not by adding a test helper), the use
of this function should be, I think, protected using appropriate test
prerequisite. Not everyone has 'dd' tool installed, for example on
MS Windows.
Second, the commit-graph file format has H-byte HASH-checksum of all of
the contents excluding checksum trailer. It feels like any corruption
should have been caught by checksum test; thus to actually test that
contents is verified we should adjust checksum too, e.g. with sha1sum if
available or with test helper... oh, actually we have t/helper/test-sha1.
Unfortulately, it looks like it has no docs (beside commit message).
> +
> +test_expect_success 'detect bad signature' '
> + cd "$TRASH_DIRECTORY/full" &&
This 'cd' outside subshell and withou accompanying change back feels a
bit strange to me.
> + cp $objdir/info/commit-graph commit-graph-backup &&
> + test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
> + corrupt_data $objdir/info/commit-graph 0 "\0" &&
So 'CGPH' signature is currupted into '\0GPH'.
> + test_must_fail git commit-graph verify 2>err &&
> + grep -v "^\+" err > verify-errors &&
Minor nit: redirection should be cuddled to the file, i.e.:
+ grep -v "^\+" err >verify-errors &&
A question: why do you filter-out lines starting with "+" here?
> + test_line_count = 1 verify-errors &&
> + grep "graph signature" verify-errors
If messages from 'git commit-graph verify' can be localized (are
translatable), then it should be i18n_grep, isn't it?
> +'
> +
> +test_expect_success 'detect bad version number' '
> + cd "$TRASH_DIRECTORY/full" &&
> + cp $objdir/info/commit-graph commit-graph-backup &&
> + test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
> + corrupt_data $objdir/info/commit-graph 4 "\02" &&
All right, so we replace commit-graph format version 1 ("\01") with
version 2 ("\02"). First, why 2 and not 0? Second, is "\02" portable?
> + test_must_fail git commit-graph verify 2>err &&
> + grep -v "^\+" err > verify-errors &&
> + test_line_count = 1 verify-errors &&
The above three lines is common across all test cases; I wonder if it
would be possible to extract it into function, to avoid code
duplication.
> + grep "graph version" verify-errors
> +'
> +
> +test_expect_success 'detect bad hash version' '
> + cd "$TRASH_DIRECTORY/full" &&
> + cp $objdir/info/commit-graph commit-graph-backup &&
> + test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
> + corrupt_data $objdir/info/commit-graph 5 "\02" &&
All right, so we change / corrupt hash version from value of 1, which
means SHA-1, to value of 2... which would soon meen NewHash. Why not
"\777" (i.e. 0xff)?
> + test_must_fail git commit-graph verify 2>err &&
> + grep -v "^\+" err > verify-errors &&
> + test_line_count = 1 verify-errors &&
> + grep "hash version" verify-errors
> +'
Note: all of the above tests check in load_commit_graph_one(), not the
one in verify_commit_graph(). Just FYI.
> +
> +test_expect_success 'detect too small chunk-count' '
> + cd "$TRASH_DIRECTORY/full" &&
> + cp $objdir/info/commit-graph commit-graph-backup &&
> + test_when_finished mv commit-graph-backup $objdir/info/commit-graph &&
> + corrupt_data $objdir/info/commit-graph 6 "\01" &&
> + test_must_fail git commit-graph verify 2>err &&
> + grep -v "^\+" err > verify-errors &&
> + test_line_count = 2 verify-errors &&
> + grep "missing the OID Lookup chunk" verify-errors &&
> + grep "missing the Commit Data chunk" verify-errors
This feels too implementation specific. We should have at least two
chunks missing (there are 3 required chunks, and number of chunks was
changed to 1), but commit-graph format specification does not state that
OID Fanout must be first, and thus it is two remaining required chunks
that would be missing.
> +'
> +
> test_done
One test that I would like to see that 'git commit-grph verify'
correctly detects without crashing is if commit-graph file gets
truncated at various lengths: shorter than smallest possible
commit-graph file size, in the middle of fixed header, in the middle of
chunk lookup part, in the middle of chunk, just the trailer chopped off.
Best regards,
--
Jakub Narębski
next prev parent reply other threads:[~2018-05-21 18:53 UTC|newest]
Thread overview: 149+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-17 18:10 [RFC PATCH 00/12] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-04-17 18:10 ` [RFC PATCH 01/12] fixup! commit-graph: always load commit-graph information Derrick Stolee
2018-04-17 18:10 ` [RFC PATCH 03/12] commit-graph: check file header information Derrick Stolee
2018-04-19 15:58 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 02/12] commit-graph: add 'check' subcommand Derrick Stolee
2018-04-19 13:24 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 04/12] commit-graph: parse commit from chosen graph Derrick Stolee
2018-04-19 17:21 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 06/12] commit: force commit to parse from object database Derrick Stolee
2018-04-20 12:13 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 05/12] commit-graph: check fanout and lookup table Derrick Stolee
2018-04-20 7:27 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 07/12] commit-graph: load a root tree from specific graph Derrick Stolee
2018-04-20 12:18 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 08/12] commit-graph: verify commit contents against odb Derrick Stolee
2018-04-20 16:47 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 10/12] commit-graph: add '--reachable' option Derrick Stolee
2018-04-20 17:17 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 09/12] fsck: check commit-graph Derrick Stolee
2018-04-20 16:59 ` Jakub Narebski
2018-04-17 18:10 ` [RFC PATCH 11/12] gc: automatically write commit-graph files Derrick Stolee
2018-04-20 17:34 ` Jakub Narebski
2018-04-20 18:33 ` Ævar Arnfjörð Bjarmason
2018-04-17 18:10 ` [RFC PATCH 12/12] commit-graph: update design document Derrick Stolee
2018-04-20 19:10 ` Jakub Narebski
2018-04-17 18:50 ` [RFC PATCH 00/12] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-05-10 17:34 ` [PATCH 00/12] Integrate commit-graph into fsck, gc, and fetch Derrick Stolee
2018-05-10 17:34 ` [PATCH 01/12] commit-graph: add 'verify' subcommand Derrick Stolee
2018-05-10 18:15 ` Martin Ågren
2018-05-10 17:34 ` [PATCH 02/12] commit-graph: verify file header information Derrick Stolee
2018-05-10 18:21 ` Martin Ågren
2018-05-10 17:34 ` [PATCH 03/12] commit-graph: parse commit from chosen graph Derrick Stolee
2018-05-10 17:34 ` [PATCH 04/12] commit-graph: verify fanout and lookup table Derrick Stolee
2018-05-10 18:29 ` Martin Ågren
2018-05-11 15:17 ` Derrick Stolee
2018-05-10 17:34 ` [PATCH 05/12] commit: force commit to parse from object database Derrick Stolee
2018-05-10 17:34 ` [PATCH 06/12] commit-graph: load a root tree from specific graph Derrick Stolee
2018-05-10 17:34 ` [PATCH 07/12] commit-graph: verify commit contents against odb Derrick Stolee
2018-05-10 17:34 ` [PATCH 08/12] fsck: verify commit-graph Derrick Stolee
2018-05-10 17:34 ` [PATCH 09/12] commit-graph: add '--reachable' option Derrick Stolee
2018-05-10 17:34 ` [PATCH 10/12] gc: automatically write commit-graph files Derrick Stolee
2018-05-10 17:34 ` [PATCH 11/12] fetch: compute commit-graph by default Derrick Stolee
2018-05-10 17:34 ` [PATCH 12/12] commit-graph: update design document Derrick Stolee
2018-05-10 19:05 ` [PATCH 00/12] Integrate commit-graph into fsck, gc, and fetch Martin Ågren
2018-05-10 19:22 ` Stefan Beller
2018-05-11 17:23 ` Derrick Stolee
2018-05-11 17:30 ` Martin Ågren
2018-05-10 19:17 ` Ævar Arnfjörð Bjarmason
2018-05-11 17:23 ` Derrick Stolee
2018-05-11 21:15 ` [PATCH v2 00/12] Integrate commit-graph into fsck and gc Derrick Stolee
2018-05-11 21:15 ` [PATCH v2 01/12] commit-graph: add 'verify' subcommand Derrick Stolee
2018-05-12 13:31 ` Martin Ågren
2018-05-14 13:27 ` Derrick Stolee
2018-05-20 12:10 ` Jakub Narebski
2018-05-11 21:15 ` [PATCH v2 02/12] commit-graph: verify file header information Derrick Stolee
2018-05-12 13:35 ` Martin Ågren
2018-05-14 13:31 ` Derrick Stolee
2018-05-20 20:00 ` Jakub Narebski
2018-05-11 21:15 ` [PATCH v2 03/12] commit-graph: test that 'verify' finds corruption Derrick Stolee
2018-05-12 13:43 ` Martin Ågren
2018-05-21 18:53 ` Jakub Narebski [this message]
2018-05-24 16:28 ` Derrick Stolee
2018-05-11 21:15 ` [PATCH v2 04/12] commit-graph: parse commit from chosen graph Derrick Stolee
2018-05-12 20:50 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 05/12] commit-graph: verify fanout and lookup table Derrick Stolee
2018-05-11 21:15 ` [PATCH v2 06/12] commit: force commit to parse from object database Derrick Stolee
2018-05-12 20:54 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 07/12] commit-graph: load a root tree from specific graph Derrick Stolee
2018-05-12 20:55 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 08/12] commit-graph: verify commit contents against odb Derrick Stolee
2018-05-12 21:17 ` Martin Ågren
2018-05-14 13:44 ` Derrick Stolee
2018-05-15 21:12 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 09/12] fsck: verify commit-graph Derrick Stolee
2018-05-17 18:13 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 11/12] gc: automatically write commit-graph files Derrick Stolee
2018-05-17 18:20 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 10/12] commit-graph: add '--reachable' option Derrick Stolee
2018-05-17 18:16 ` Martin Ågren
2018-05-11 21:15 ` [PATCH v2 12/12] commit-graph: update design document Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 00/20] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 01/20] commit-graph: UNLEAK before die() Derrick Stolee
2018-05-24 22:47 ` Stefan Beller
2018-05-25 0:08 ` Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 02/20] commit-graph: fix GRAPH_MIN_SIZE Derrick Stolee
2018-05-26 18:46 ` Jakub Narebski
2018-05-26 20:30 ` brian m. carlson
2018-06-02 19:43 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 03/20] commit-graph: parse commit from chosen graph Derrick Stolee
2018-05-27 10:23 ` Jakub Narebski
2018-05-29 12:31 ` Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 04/20] commit: force commit to parse from object database Derrick Stolee
2018-05-27 18:04 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 05/20] commit-graph: load a root tree from specific graph Derrick Stolee
2018-05-27 19:12 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 06/20] commit-graph: add 'verify' subcommand Derrick Stolee
2018-05-27 22:55 ` Jakub Narebski
2018-05-30 16:07 ` Derrick Stolee
2018-06-02 21:19 ` Jakub Narebski
2018-06-04 11:30 ` Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 07/20] commit-graph: verify catches corrupt signature Derrick Stolee
2018-05-28 14:05 ` Jakub Narebski
2018-05-29 12:43 ` Derrick Stolee
2018-06-02 22:30 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 08/20] commit-graph: verify required chunks are present Derrick Stolee
2018-05-28 17:11 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 09/20] commit-graph: verify corrupt OID fanout and lookup Derrick Stolee
2018-05-30 13:34 ` Jakub Narebski
2018-05-30 16:18 ` Derrick Stolee
2018-06-02 4:38 ` Duy Nguyen
2018-06-04 11:32 ` Derrick Stolee
2018-06-04 14:42 ` Duy Nguyen
2018-05-24 16:25 ` [PATCH v3 10/20] commit-graph: verify objects exist Derrick Stolee
2018-05-30 19:22 ` Jakub Narebski
2018-05-31 12:53 ` Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 11/20] commit-graph: verify root tree OIDs Derrick Stolee
2018-05-30 22:24 ` Jakub Narebski
2018-05-31 13:16 ` Derrick Stolee
2018-06-02 22:50 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 12/20] commit-graph: verify parent list Derrick Stolee
2018-06-01 23:21 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 13/20] commit-graph: verify generation number Derrick Stolee
2018-06-02 12:23 ` Jakub Narebski
2018-06-04 11:47 ` Derrick Stolee
2018-05-24 16:25 ` [PATCH v3 14/20] commit-graph: verify commit date Derrick Stolee
2018-06-02 12:29 ` Jakub Narebski
2018-05-24 16:25 ` [PATCH v3 15/20] commit-graph: test for corrupted octopus edge Derrick Stolee
2018-06-02 12:39 ` Jakub Narebski
2018-06-04 13:08 ` Derrick Stolee
2018-05-24 16:26 ` [PATCH v3 16/20] commit-graph: verify contents match checksum Derrick Stolee
2018-05-30 12:35 ` SZEDER Gábor
2018-06-02 15:52 ` Jakub Narebski
2018-06-04 11:55 ` Derrick Stolee
2018-05-24 16:26 ` [PATCH v3 17/20] fsck: verify commit-graph Derrick Stolee
2018-06-02 16:17 ` Jakub Narebski
2018-06-04 11:59 ` Derrick Stolee
2018-05-24 16:26 ` [PATCH v3 18/20] commit-graph: add '--reachable' option Derrick Stolee
2018-06-02 17:34 ` Jakub Narebski
2018-06-04 12:44 ` Derrick Stolee
2018-05-24 16:26 ` [PATCH v3 19/20] gc: automatically write commit-graph files Derrick Stolee
2018-06-02 18:03 ` Jakub Narebski
2018-06-04 12:51 ` Derrick Stolee
2018-05-24 16:26 ` [PATCH v3 20/20] commit-graph: update design document Derrick Stolee
2018-06-02 18:27 ` Jakub Narebski
2018-05-24 21:15 ` [PATCH v3 00/20] Integrate commit-graph into 'fsck' and 'gc' Ævar Arnfjörð Bjarmason
2018-05-25 4:11 ` Junio C Hamano
2018-05-29 4:27 ` Junio C Hamano
2018-05-29 12:37 ` Derrick Stolee
2018-05-29 13:41 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=86wovwdemu.fsf@gmail.com \
--to=jnareb@gmail.com \
--cc=avarab@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=martin.agren@gmail.com \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).