[PATCH] packfile: use get_be64() for large offsets

[PATCH] packfile: use get_be64() for large offsets

[Derrick Stolee]

The pack-index version 2 format uses two 4-byte integers in network-byte order to represent one 8-byte value. The current implementation has several code clones for stitching these integers together.

Use get_be64() to create an 8-byte integer from two 4-byte integers represented this way.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 pack-revindex.c | 6 ++----
 packfile.c      | 3 +--
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/pack-revindex.c b/pack-revindex.c
index 1b7ebd8d7e..ff5f62c033 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -134,10 +134,8 @@ static void create_pack_revindex(struct packed_git *p)
 			if (!(off & 0x80000000)) {
 				p->revindex[i].offset = off;
 			} else {
-				p->revindex[i].offset =
-					((uint64_t)ntohl(*off_64++)) << 32;
-				p->revindex[i].offset |=
-					ntohl(*off_64++);
+				p->revindex[i].offset = get_be64(off_64);
+				off_64 += 2;
 			}
 			p->revindex[i].nr = i;
 		}
diff --git a/packfile.c b/packfile.c
index 4a5fe7ab18..228ed0d59a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1702,8 +1702,7 @@ off_t nth_packed_object_offset(const struct packed_git *p, uint32_t n)
 			return off;
 		index += p->num_objects * 4 + (off & 0x7fffffff) * 8;
 		check_pack_index_ptr(p, index);
-		return (((uint64_t)ntohl(*((uint32_t *)(index + 0)))) << 32) |
-				   ntohl(*((uint32_t *)(index + 4)));
+		return get_be64(index);
 	}
 }
 
-- 
2.15.0

Re: [PATCH] packfile: use get_be64() for large offsets

[Jeff King]

On Wed, Jan 17, 2018 at 02:08:23PM -0500, Derrick Stolee wrote:

> The pack-index version 2 format uses two 4-byte integers in network-byte order to represent one 8-byte value. The current implementation has several code clones for stitching these integers together.

Yeah, this seems like a no-brainer, and the patch itself looks obviously
correct.

Please wrap your commit messages a bit shorter, though (say, between
60-72 chars).

-Peff

[PATCH v2] packfile: use get_be64() for large offsets

[Derrick Stolee]

The pack-index version 2 format uses two 4-byte integers in network-
byte order to represent one 8-byte value. The current implementation
has several code clones for stitching these integers together.

Use get_be64() to create an 8-byte integer from two 4-byte integers
represented this way.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 pack-revindex.c | 6 ++----
 packfile.c      | 3 +--
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/pack-revindex.c b/pack-revindex.c
index 1b7ebd8d7e..ff5f62c033 100644
--- a/pack-revindex.c
+++ b/pack-revindex.c
@@ -134,10 +134,8 @@ static void create_pack_revindex(struct packed_git *p)
 			if (!(off & 0x80000000)) {
 				p->revindex[i].offset = off;
 			} else {
-				p->revindex[i].offset =
-					((uint64_t)ntohl(*off_64++)) << 32;
-				p->revindex[i].offset |=
-					ntohl(*off_64++);
+				p->revindex[i].offset = get_be64(off_64);
+				off_64 += 2;
 			}
 			p->revindex[i].nr = i;
 		}
diff --git a/packfile.c b/packfile.c
index 4a5fe7ab18..228ed0d59a 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1702,8 +1702,7 @@ off_t nth_packed_object_offset(const struct packed_git *p, uint32_t n)
 			return off;
 		index += p->num_objects * 4 + (off & 0x7fffffff) * 8;
 		check_pack_index_ptr(p, index);
-		return (((uint64_t)ntohl(*((uint32_t *)(index + 0)))) << 32) |
-				   ntohl(*((uint32_t *)(index + 4)));
+		return get_be64(index);
 	}
 }
 
-- 
2.15.0

Re: [PATCH v2] packfile: use get_be64() for large offsets

[SZEDER Gábor]

Junio,

> The pack-index version 2 format uses two 4-byte integers in network-
> byte order to represent one 8-byte value. The current implementation
> has several code clones for stitching these integers together.
> 
> Use get_be64() to create an 8-byte integer from two 4-byte integers
> represented this way.
> 
> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
> ---
>  pack-revindex.c | 6 ++----
>  packfile.c      | 3 +--
>  2 files changed, 3 insertions(+), 6 deletions(-)
> 
> diff --git a/pack-revindex.c b/pack-revindex.c
> index 1b7ebd8d7e..ff5f62c033 100644
> --- a/pack-revindex.c
> +++ b/pack-revindex.c
> @@ -134,10 +134,8 @@ static void create_pack_revindex(struct packed_git *p)
>  			if (!(off & 0x80000000)) {
>  				p->revindex[i].offset = off;
>  			} else {
> -				p->revindex[i].offset =
> -					((uint64_t)ntohl(*off_64++)) << 32;
> -				p->revindex[i].offset |=
> -					ntohl(*off_64++);
> +				p->revindex[i].offset = get_be64(off_64);
> +				off_64 += 2;
>  			}
>  			p->revindex[i].nr = i;
>  		}
> diff --git a/packfile.c b/packfile.c
> index 4a5fe7ab18..228ed0d59a 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -1702,8 +1702,7 @@ off_t nth_packed_object_offset(const struct packed_git *p, uint32_t n)
>  			return off;
>  		index += p->num_objects * 4 + (off & 0x7fffffff) * 8;
>  		check_pack_index_ptr(p, index);
> -		return (((uint64_t)ntohl(*((uint32_t *)(index + 0)))) << 32) |
> -				   ntohl(*((uint32_t *)(index + 4)));
> +		return get_be64(index);
>  	}
>  }
>  
> -- 
> 2.15.0
> 

This patch can't be applied to 'maint' currently at 3013dff86 (Prepare
for 2.15.2, 2017-12-06), as it is in case of 'ds/use-get-be64',
because 'maint' doesn't have get_be64() yet (b2e39d006 (bswap: add 64
bit endianness helper get_be64, 2017-09-22)).


Gábor

Re: [PATCH v2] packfile: use get_be64() for large offsets

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> Junio,
> ...
> This patch can't be applied to 'maint' currently at 3013dff86 (Prepare
> for 2.15.2, 2017-12-06), as it is in case of 'ds/use-get-be64',
> because 'maint' doesn't have get_be64() yet (b2e39d006 (bswap: add 64
> bit endianness helper get_be64, 2017-09-22)).

Thanks for stopping me.  

I do not always get to test anything not in 'next' in isolation
before I actually try to merge it there (at which time I would have
noticed), and you caught me in this one doing a short-cut "just
queue it somewhere and throw it in 'pu'".