From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: Taylor Blau <me@ttaylorr.com>
Subject: [PATCH 0/20] bounds-checks for chunk-based files
Date: Mon, 9 Oct 2023 16:55:44 -0400 [thread overview]
Message-ID: <20231009205544.GA3281950@coredump.intra.peff.net> (raw)
As part of my -Wunused-parameter series, I noticed that a few callbacks
used with the chunk-format API ignored the "chunk_size" parameter. I had
initially just annotated these with UNUSED under the usual "well, not
all callbacks need all parameters" logic.
But if you think about it, a chunk callback that does not look at the
chunk size _must_ be buggy, as there is no way it could ensure that it
does not read past the end of the chunk. In a well-formed file this
isn't a problem, since most chunks have expected sizes (e.g., a
commit-graph has a fixed-size commit-data record for every commit
mentioned in its index chunk). But it is very easy to get out-of-bounds
reads for files that are not well-formed.
I think the security implications here are pretty minor. The only files
which use the chunk format are multi-pack-index and commit-graph,
neither of which we'd expect to receive over the network (so you'd need
to access an untrusted tarball, etc, to even see a malicious file). And
we'd never try to write to this memory (they're read-only mmaps of the
files). So your worst case is probably an unexpected out-of-bounds read
and segfault, on a file that is hard to put in front of the victim.
But I think it's still worth fixing. The extra checks aren't very
expensive, and let us handle bugs or corruption more gracefully, as
well.
It's a lot of patches because there are a lot of chunk types. ;) But
each one is hopefully pretty straightforward in isolation. I tried to
group similar chunks together (e.g., commit-graph and midx both have
OIDF and OIDL chunks), but otherwise just fixed the midx chunks in the
order they appear in the code, followed by the same for commit-graph.
[01/20]: chunk-format: note that pair_chunk() is unsafe
[02/20]: t: add library for munging chunk-format files
[03/20]: midx: stop ignoring malformed oid fanout chunk
[04/20]: commit-graph: check size of oid fanout chunk
[05/20]: midx: check size of oid lookup chunk
[06/20]: commit-graph: check consistency of fanout table
[07/20]: midx: check size of pack names chunk
[08/20]: midx: enforce chunk alignment on reading
[09/20]: midx: check size of object offset chunk
[10/20]: midx: bounds-check large offset chunk
[11/20]: midx: check size of revindex chunk
[12/20]: commit-graph: check size of commit data chunk
[13/20]: commit-graph: detect out-of-bounds extra-edges pointers
[14/20]: commit-graph: bounds-check base graphs chunk
[15/20]: commit-graph: check size of generations chunk
[16/20]: commit-graph: bounds-check generation overflow chunk
[17/20]: commit-graph: check bounds when accessing BDAT chunk
[18/20]: commit-graph: check bounds when accessing BIDX chunk
[19/20]: commit-graph: detect out-of-order BIDX offsets
[20/20]: chunk-format: drop pair_chunk_unsafe()
bloom.c | 34 +++++++++
chunk-format.c | 24 ++++--
chunk-format.h | 9 ++-
commit-graph.c | 119 ++++++++++++++++++++++++-----
commit-graph.h | 4 +
midx.c | 68 +++++++++++++----
midx.h | 3 +
pack-revindex.c | 13 +++-
t/lib-chunk.sh | 17 +++++
t/lib-chunk/corrupt-chunk-file.pl | 66 ++++++++++++++++
t/t4216-log-bloom.sh | 50 ++++++++++++
t/t5318-commit-graph.sh | 76 +++++++++++++++++-
t/t5319-multi-pack-index.sh | 102 ++++++++++++++++++++++++-
t/t5324-split-commit-graph.sh | 20 ++++-
t/t5328-commit-graph-64bit-time.sh | 10 +++
15 files changed, 568 insertions(+), 47 deletions(-)
create mode 100644 t/lib-chunk.sh
create mode 100644 t/lib-chunk/corrupt-chunk-file.pl
-Peff
next reply other threads:[~2023-10-09 20:56 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-09 20:55 Jeff King [this message]
2023-10-09 20:58 ` [PATCH 01/20] chunk-format: note that pair_chunk() is unsafe Jeff King
2023-10-10 23:45 ` Taylor Blau
2023-10-11 22:49 ` Jeff King
2023-10-09 20:58 ` [PATCH 02/20] t: add library for munging chunk-format files Jeff King
2023-10-10 23:47 ` Taylor Blau
2023-10-09 20:59 ` [PATCH 03/20] midx: stop ignoring malformed oid fanout chunk Jeff King
2023-10-10 23:50 ` Taylor Blau
2023-10-11 22:52 ` Jeff King
2023-10-09 20:59 ` [PATCH 04/20] commit-graph: check size of " Jeff King
2023-10-11 0:08 ` Taylor Blau
2023-10-11 1:24 ` Taylor Blau
2023-10-11 23:01 ` Jeff King
2023-10-09 21:02 ` [PATCH 05/20] midx: check size of oid lookup chunk Jeff King
2023-10-09 21:04 ` [PATCH 06/20] commit-graph: check consistency of fanout table Jeff King
2023-10-11 14:45 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 07/20] midx: check size of pack names chunk Jeff King
2023-10-11 14:52 ` Taylor Blau
2023-10-11 23:06 ` Jeff King
2023-10-09 21:05 ` [PATCH 08/20] midx: enforce chunk alignment on reading Jeff King
2023-10-11 14:56 ` Taylor Blau
2023-10-11 15:01 ` Taylor Blau
2023-10-11 23:09 ` Jeff King
2023-10-09 21:05 ` [PATCH 09/20] midx: check size of object offset chunk Jeff King
2023-10-11 18:31 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 10/20] midx: bounds-check large " Jeff King
2023-10-11 18:38 ` Taylor Blau
2023-10-11 23:18 ` Jeff King
2023-10-09 21:05 ` [PATCH 11/20] midx: check size of revindex chunk Jeff King
2023-10-11 18:41 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 12/20] commit-graph: check size of commit data chunk Jeff King
2023-10-11 18:46 ` Taylor Blau
2023-10-11 23:22 ` Jeff King
2023-10-09 21:05 ` [PATCH 13/20] commit-graph: detect out-of-bounds extra-edges pointers Jeff King
2023-10-11 19:02 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 14/20] commit-graph: bounds-check base graphs chunk Jeff King
2023-10-11 19:05 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 15/20] commit-graph: check size of generations chunk Jeff King
2023-10-09 21:05 ` [PATCH 16/20] commit-graph: bounds-check generation overflow chunk Jeff King
2023-10-09 21:05 ` [PATCH 17/20] commit-graph: check bounds when accessing BDAT chunk Jeff King
2023-10-11 19:11 ` Taylor Blau
2023-10-11 23:27 ` Jeff King
2023-10-09 21:05 ` [PATCH 18/20] commit-graph: check bounds when accessing BIDX chunk Jeff King
2023-10-11 19:15 ` Taylor Blau
2023-10-09 21:05 ` [PATCH 19/20] commit-graph: detect out-of-order BIDX offsets Jeff King
2023-10-11 19:16 ` Taylor Blau
2023-10-09 21:06 ` [PATCH 20/20] chunk-format: drop pair_chunk_unsafe() Jeff King
2023-10-11 19:19 ` [PATCH 0/20] bounds-checks for chunk-based files Taylor Blau
2023-10-11 23:31 ` Jeff King
2023-10-13 19:25 ` [PATCH 0/8] chunk-format: introduce `pair_chunk_expect()` convenience API Taylor Blau
2023-10-13 19:25 ` [PATCH 1/8] chunk-format: introduce `pair_chunk_expect()` helper Taylor Blau
2023-10-13 19:25 ` [PATCH 2/8] commit-graph: read `OIDF` chunk with `pair_chunk_expect()` Taylor Blau
2023-10-13 19:25 ` [PATCH 3/8] commit-graph: read `CDAT` " Taylor Blau
2023-10-13 19:25 ` [PATCH 4/8] commit-graph: read `GDAT` " Taylor Blau
2023-10-13 19:25 ` [PATCH 5/8] commit-graph: read `BIDX` " Taylor Blau
2023-10-13 19:49 ` Taylor Blau
2023-10-14 16:10 ` Junio C Hamano
2023-10-20 10:31 ` Jeff King
2023-10-13 19:25 ` [PATCH 6/8] midx: read `OIDF` " Taylor Blau
2023-10-13 21:04 ` Junio C Hamano
2023-10-13 19:25 ` [PATCH 7/8] midx: read `OIDL` " Taylor Blau
2023-10-13 19:25 ` [PATCH 8/8] midx: read `OOFF` " Taylor Blau
2023-10-20 10:23 ` [PATCH 0/8] chunk-format: introduce `pair_chunk_expect()` convenience API Jeff King
2023-10-14 0:43 ` [PATCH 21/20] t5319: make corrupted large-offset test more robust Jeff King
2023-10-14 19:42 ` Junio C Hamano
2023-10-15 3:17 ` Jeff King
2023-10-15 17:04 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231009205544.GA3281950@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=me@ttaylorr.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).