From: Junio C Hamano <gitster@pobox.com>
To: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 2/2] index-pack: use streaming interface on large blobs (most of the time)
Date: Fri, 18 May 2012 15:20:15 -0700 [thread overview]
Message-ID: <xmqqzk95168w.fsf@junio.mtv.corp.google.com> (raw)
In-Reply-To: <1337172630-23679-2-git-send-email-pclouds@gmail.com> ("Nguyễn Thái Ngọc Duy"'s message of "Wed, 16 May 2012 19:50:30 +0700")
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> unpack_raw_entry() will not allocate and return decompressed blobs if
> they are larger than core.bigFileThreshold.
>
> The blob content is needed by sha1_object() in some cases. When we do
> need the blob content, we put it back in core with
> get_data_from_pack(). However we rarely need that in pratice.
>
> The first case is when we find an in-repo blob with the same SHA-1. We
> need to do collision test, byte-on-byte. Normally (e.g. in
> fetch/pull/clone) this does not happen because git avoid to send
> objects that client already has.
Perhaps the codepath that performs the byte-for-byte comparison can be
taught to stream from the received pack data and whatever was already
in the repository, using the streaming interface? That way you do not
have to hold all of the both objects at the same time in core, no?
next prev parent reply other threads:[~2012-05-18 22:20 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-05-16 12:50 [PATCH 1/2] index-pack: hash non-delta objects while reading from stream Nguyễn Thái Ngọc Duy
2012-05-16 12:50 ` [PATCH 2/2] index-pack: use streaming interface on large blobs (most of the time) Nguyễn Thái Ngọc Duy
2012-05-18 22:20 ` Junio C Hamano [this message]
2012-05-19 5:31 ` Nguyen Thai Ngoc Duy
2012-05-18 22:15 ` [PATCH 1/2] index-pack: hash non-delta objects while reading from stream Junio C Hamano
2012-05-23 14:09 ` [PATCH v2 1/6] " Nguyễn Thái Ngọc Duy
2012-05-23 14:09 ` [PATCH v2 2/6] index-pack: use streaming interface on large blobs (most of the time) Nguyễn Thái Ngọc Duy
2012-05-23 14:09 ` [PATCH v2 3/6] index-pack: factor out unpack core from get_data_from_pack Nguyễn Thái Ngọc Duy
2012-05-23 14:09 ` [PATCH v2 4/6] index-pack: use streaming interface for collision test on large blobs Nguyễn Thái Ngọc Duy
2012-05-23 16:03 ` Junio C Hamano
2012-05-24 13:55 ` [PATCH v2.1 " Nguyễn Thái Ngọc Duy
2012-05-23 14:09 ` [PATCH v2 5/6] index-pack: avoid collision test when verifying in-repo pack Nguyễn Thái Ngọc Duy
2012-05-23 14:09 ` [PATCH v2 6/6] sha1_loose_object_info: do not complain out loud on non-existent objects Nguyễn Thái Ngọc Duy
2012-05-23 14:24 ` Nguyen Thai Ngoc Duy
2012-05-23 16:01 ` Junio C Hamano
2012-05-24 13:12 ` Nguyen Thai Ngoc Duy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqzk95168w.fsf@junio.mtv.corp.google.com \
--to=gitster@pobox.com \
--cc=git@vger.kernel.org \
--cc=pclouds@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).