Understanding Binary Deltas within Packfile

Understanding Binary Deltas within Packfile

[Luke Robison]

Is there any documentation of the contents of the binary delta datain a 
packfile, and how to interpret them?  I found 
https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt 
documenting the packfile itself, but the "compressed delta data" seems 
largely undocumented.  The source code of 
https://github.com/git/git/blob/master/diff-delta.c is pretty dense.  
This is what I've got so for:

+-------------+
| Varint src_size
| Varint trg_size
| 1-byte inscnt
| N-bytes up to 16 bytes of trg_buf
|
| A series of Operations
|
+-------------


Any details on how to interpret the opcodes would be greatly appreciated.

Thanks,
Luke

Re: Understanding Binary Deltas within Packfile

[Duy Nguyen]

On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
> Is there any documentation of the contents of the binary delta datain a
> packfile, and how to interpret them?  I found
> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
> documenting the packfile itself, but the "compressed delta data" seems
> largely undocumented.  The source code of
> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.

The output is consumed by patch_delta()  in patch-delta.c if I'm not
mistaken. This function is less than 100 lines, probably much easier
to see the delta format.

>  This
> is what I've got so for:
>
> +-------------+
> | Varint src_size
> | Varint trg_size
> | 1-byte inscnt
> | N-bytes up to 16 bytes of trg_buf
> |
> | A series of Operations
> |
> +-------------
>
>
> Any details on how to interpret the opcodes would be greatly appreciated.
>
> Thanks,
> Luke
>



-- 
Duy

Re: Understanding Binary Deltas within Packfile

[Luke Robison]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On 3/20/2018 11:03 AM, Duy Nguyen wrote:
> On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
>> Is there any documentation of the contents of the binary delta datain a
>> packfile, and how to interpret them?  I found
>> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
>> documenting the packfile itself, but the "compressed delta data" seems
>> largely undocumented.  The source code of
>> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.
> The output is consumed by patch_delta()  in patch-delta.c if I'm not
> mistaken. This function is less than 100 lines, probably much easier
> to see the delta format.
Thank you, that was much easier to read, and I've got my prototype 
working now.  I also found this site to be quite helpful:
http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding




[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4311 bytes --]

Re: Understanding Binary Deltas within Packfile

[Duy Nguyen]

On Tue, Mar 20, 2018 at 7:34 PM, Luke Robison <robison@arlut.utexas.edu> wrote:
> On 3/20/2018 11:03 AM, Duy Nguyen wrote:
>>
>> On Tue, Mar 20, 2018 at 4:43 PM, Luke Robison <robison@arlut.utexas.edu>
>> wrote:
>>>
>>> Is there any documentation of the contents of the binary delta datain a
>>> packfile, and how to interpret them?  I found
>>>
>>> https://github.com/git/git/blob/master/Documentation/technical/pack-format.txt
>>> documenting the packfile itself, but the "compressed delta data" seems
>>> largely undocumented.  The source code of
>>> https://github.com/git/git/blob/master/diff-delta.c is pretty dense.
>>
>> The output is consumed by patch_delta()  in patch-delta.c if I'm not
>> mistaken. This function is less than 100 lines, probably much easier
>> to see the delta format.
>
> Thank you, that was much easier to read, and I've got my prototype working
> now.  I also found this site to be quite helpful:
> http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/#delta_encoding

By the way, I forgot to add, if you want to improve pack-format.txt
(since you study it anyway), patches are always welcome.
-- 
Duy