Re: pack file object size question

[Jonathan Nieder <jrnieder@gmail.com>]

Hi,

Farhan Khan wrote:
>> Farhan Khan wrote:

>>> I am having trouble figuring out the boundary between two objects in
>>> the pack file.
[...]
>              I think the issue is, the compressed object has a fixed
> size and git inflates it, then moves on to the next object. I am
> trying to figure out how where it identifies the size of the object.

Do you mean the compressed size or uncompressed size?

It sounds to me like pack-format.txt needs to do a better job of
distinguishing the two.  Under "Pack file entry", I see

| Pack file entry: <+
|
|    packed object header:
|	1-byte size extension bit (MSB)
|	       type (next 3 bit)
|	       size0 (lower 4-bit)
|	n-byte sizeN (as long as MSB is set, each 7-bit)
|		size0..sizeN form 4+7+7+..+7 bit integer, size0
|		is the least significant part, and sizeN is the
|		most significant part.
|    packed object data:
|	If it is not DELTA, then deflated bytes (the size above
|		is the size before compression).
|	If it is REF_DELTA, then
|	  20-byte base object name SHA-1 (the size above is the
|		size of the delta data that follows).
|	  delta data, deflated.
|	If it is OFS_DELTA, then
|	  n-byte offset (see below) interpreted as a negative
|		offset from the type-byte of the header of the
|		ofs-delta entry (the size above is the size of
|		the delta data that follows).
|	  delta data, deflated.

which suggests that the "length" field is something between the two:
it is the size of the inflated form of the packed object data, before
resolving deltas.  It's useful for allocating a buffer to inflate
into.

The zlib container format (https://tools.ietf.org/html/rfc1950) does
not contain size information, so I believe you'll have to use a
"deflate" (https://tools.ietf.org/html/rfc1951) decoder such as zlib
to find the end of the deflated bytes.

In index-pack, you need to inflate the objects anyway.  In random
lookups, the idx file tells you where to look, so it doesn't come up
there, either.  So this would only be expected to come up if you are
doing a sort of partial index-pack that wants to skip some objects.

Thanks and hope that helps,
Jonathan