Re: Confused over packfile and index design

["Steven E. Harris" <seh@panix.com>]

Shawn Pearce <spearce@spearce.org> writes:

> Then remaining bytes are shoved into a libz inflate() routine until
> libz says the stream is over. As Peff mentioned elsewhere in the
> thread, libz maintains its own markers and checksum to know when the
> object's stream is over.

Ah, so even though you as the caller don't know how much data to feed to
libz, so long as you continue feeding it until it signals completion, it
will figure it out and tell you how much data it needed after all.

> As a safety measure, the inflated length from the object header is
> checked against the number of bytes returned by libz.  Any remaining
> data that libz didn't consume is the next object's header and data.

I see. This means that it's the packed object's "job" -- or, rather, the
job of the parser for the packed object -- to determine the payload
length. If the data was not compressed, then perhaps the deflated size
indicated in the header could provide sufficient framing, but for now we
don't need to worry about such flexibility.

[...]

> Its meant to tell us how many bytes to malloc() in order to hold the
> result of the libz inflate() call when the object is being read from
> the packfile. That way we don't under or over allocate the result
> buffer.

Does Git always inflate the objects into an in-memory buffer? As the
size of these objects can be very large (given the variable-length size
encoding), is there any provision to inflate the object to a temporary
file?

-- 
Steven E. Harris