From: Junio C Hamano <gitster@pobox.com>
To: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Cc: git@vger.kernel.org, michelbach94@gmail.com
Subject: Re: [PATCH 7/5] fsck: use streaming interface for large blobs in pack
Date: Tue, 12 Jul 2016 12:06:32 -0700 [thread overview]
Message-ID: <xmqqvb0a4qx3.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <xmqq7fcq66q6.fsf@gitster.mtv.corp.google.com> (Junio C. Hamano's message of "Tue, 12 Jul 2016 11:39:45 -0700")
Junio C Hamano <gitster@pobox.com> writes:
>> + if (type != OBJ_BLOB || size < big_file_threshold) {
>> + data = unpack_entry(p, entries[i].offset, &type, &size);
>> + data_valid = 1;
>
> This codepath slurps the data in-core to hash and data is later
> freed, i.e. non-blob objects and small ones are handled as before.
>
>> + }
>> +
>> + if (data_valid && !data)
>> err = error("cannot unpack %s from %s at offset %"PRIuMAX"",
>> sha1_to_hex(entries[i].sha1), p->pack_name,
>> (uintmax_t)entries[i].offset);
>
> Otherwise, we'd go to check_sha1_signature() with map==NULL. And
> that is exactly what we want---map==NULL is the way we tell the
> function to use the streaming interface to check.
>
> Good.
But not quite correct yet ;-)
Here is what I'd propose to squash in. Even though data_valid
protects the above if() condition from accessing an otherwise
uninitialized "data", the call to check_sha1_signature() we have
later will get noticed by the compiler that "data" is passed
uninitialized.
More importantly, uninitialized data passed may be non-NULL, in
which case it would not trigger the streaming interface.
pack-check.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/pack-check.c b/pack-check.c
index 0777766..ac263fd 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -123,7 +123,14 @@ static int verify_packfile(struct packed_git *p,
type = unpack_object_header(p, w_curs, &curpos, &size);
unuse_pack(w_curs);
- if (type != OBJ_BLOB || size < big_file_threshold) {
+ if (type == OBJ_BLOB && big_file_threshold <= size) {
+ /*
+ * Let check_sha1_signature() to check it with
+ * the streaming interface; no point slurping
+ * the data in-core only to discard.
+ */
+ data = NULL;
+ } else {
data = unpack_entry(p, entries[i].offset, &type, &size);
data_valid = 1;
}
next prev parent reply other threads:[~2016-07-12 19:07 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-06-24 22:38 [bug] Reliably Reproducible Bad Packing of Objects Christoph Michelbach
2016-07-02 9:10 ` Duy Nguyen
2016-07-02 14:35 ` Duy Nguyen
2016-07-05 17:05 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Nguyễn Thái Ngọc Duy
2016-07-05 17:05 ` [PATCH 1/5] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-12 17:16 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 2/5] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-12 17:20 ` Junio C Hamano
2016-07-12 19:26 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 3/5] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-05 20:25 ` Johannes Sixt
2016-07-06 15:25 ` Duy Nguyen
2016-07-06 16:04 ` Junio C Hamano
2016-07-06 16:08 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 4/5] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-12 17:24 ` Junio C Hamano
2016-07-12 19:27 ` Junio C Hamano
2016-07-05 17:05 ` [PATCH 5/5] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-05 18:11 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Christoph Michelbach
[not found] ` <1467756891.4798.1.camel@gmail.com>
[not found] ` <CACsJy8BDQbanGsf=3z3K-OuH0++EuqQFEB22udXJT+WZnFKSBg@mail.gmail.com>
2016-07-06 18:02 ` Christoph Michelbach
2016-07-06 18:54 ` Duy Nguyen
2016-07-10 10:41 ` Duy Nguyen
2016-07-10 10:42 ` [PATCH 6/5] pack-objects: do not truncate result in-pack object size " Nguyễn Thái Ngọc Duy
2016-07-10 10:45 ` [PATCH 7/5] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-12 18:39 ` Junio C Hamano
2016-07-12 19:06 ` Junio C Hamano [this message]
2016-07-12 17:07 ` [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
2016-07-12 18:48 ` Junio C Hamano
2016-07-12 20:38 ` Junio C Hamano
2016-07-13 6:01 ` Duy Nguyen
2016-07-13 15:43 ` [PATCH v2 0/7] " Nguyễn Thái Ngọc Duy
2016-07-13 15:43 ` [PATCH v2 1/7] pack-objects: pass length to check_pack_crc() without truncation Nguyễn Thái Ngọc Duy
2016-07-13 15:43 ` [PATCH v2 2/7] sha1_file.c: use type off_t* for object_info->disk_sizep Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 3/7] index-pack: correct "len" type in unpack_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 4/7] index-pack: report correct bad object offsets even if they are large Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 5/7] index-pack: correct "offset" type in unpack_entry_data() Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 6/7] pack-objects: do not truncate result in-pack object size on 32-bit systems Nguyễn Thái Ngọc Duy
2016-07-13 15:44 ` [PATCH v2 7/7] fsck: use streaming interface for large blobs in pack Nguyễn Thái Ngọc Duy
2016-07-13 16:16 ` [PATCH v2 0/7] Number truncation with 4+ GB files on 32-bit systems Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqvb0a4qx3.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox.com \
--cc=git@vger.kernel.org \
--cc=michelbach94@gmail.com \
--cc=pclouds@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).