git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH 1/3] csum-file: add hashwrite_be64()
@ 2020-11-12 12:20 René Scharfe
  2020-11-12 12:22 ` [PATCH 2/3] midx: use hashwrite_be64() René Scharfe
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: René Scharfe @ 2020-11-12 12:20 UTC (permalink / raw)
  To: Git Mailing List; +Cc: Junio C Hamano, Taylor Blau, Derrick Stolee

Add a helper function for hashing and writing 64-bit integers in network
byte order.  It returns the number of written bytes.  This simplifies
callers that keep track of the file offset, even though this number is a
constant.

Suggested-by: Derrick Stolee <dstolee@microsoft.com>
Original-patch-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
---
 csum-file.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/csum-file.h b/csum-file.h
index f9cbd317fb..e54d53d1d0 100644
--- a/csum-file.h
+++ b/csum-file.h
@@ -62,4 +62,11 @@ static inline void hashwrite_be32(struct hashfile *f, uint32_t data)
 	hashwrite(f, &data, sizeof(data));
 }

+static inline size_t hashwrite_be64(struct hashfile *f, uint64_t data)
+{
+	data = htonll(data);
+	hashwrite(f, &data, sizeof(data));
+	return sizeof(data);
+}
+
 #endif
--
2.29.2

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 2/3] midx: use hashwrite_be64()
  2020-11-12 12:20 [PATCH 1/3] csum-file: add hashwrite_be64() René Scharfe
@ 2020-11-12 12:22 ` René Scharfe
  2020-11-12 12:23 ` [PATCH 3/3] pack-write: " René Scharfe
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: René Scharfe @ 2020-11-12 12:22 UTC (permalink / raw)
  To: Git Mailing List; +Cc: Junio C Hamano, Taylor Blau, Derrick Stolee

Call hashwrite_be64() to write 64-bit values instead of open-coding it
using hashwrite_be32() and sizeof.  This shortens the code and makes its
intent clearer.

Signed-off-by: René Scharfe <l.s.r@web.de>
---
And it saves me from headache induced by trying to remember whether "big
endian" means that the most significant ("big") digit comes first (yes)
or at the end, like the name suggests (no).

 midx.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/midx.c b/midx.c
index d233b54ac7..da03c1449a 100644
--- a/midx.c
+++ b/midx.c
@@ -785,9 +785,7 @@ static size_t write_midx_large_offsets(struct hashfile *f, uint32_t nr_large_off
 		if (!(offset >> 31))
 			continue;

-		hashwrite_be32(f, offset >> 32);
-		hashwrite_be32(f, offset & 0xffffffffUL);
-		written += 2 * sizeof(uint32_t);
+		written += hashwrite_be64(f, offset);

 		nr_large_offset--;
 	}
@@ -975,8 +973,7 @@ static int write_midx_internal(const char *object_dir, struct multi_pack_index *
 			    chunk_offsets[i]);

 		hashwrite_be32(f, chunk_ids[i]);
-		hashwrite_be32(f, chunk_offsets[i] >> 32);
-		hashwrite_be32(f, chunk_offsets[i]);
+		hashwrite_be64(f, chunk_offsets[i]);

 		written += MIDX_CHUNKLOOKUP_WIDTH;
 	}
--
2.29.2

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 3/3] pack-write: use hashwrite_be64()
  2020-11-12 12:20 [PATCH 1/3] csum-file: add hashwrite_be64() René Scharfe
  2020-11-12 12:22 ` [PATCH 2/3] midx: use hashwrite_be64() René Scharfe
@ 2020-11-12 12:23 ` René Scharfe
  2020-11-12 13:52 ` [PATCH 1/3] csum-file: add hashwrite_be64() Derrick Stolee
  2020-11-12 14:51 ` Taylor Blau
  3 siblings, 0 replies; 6+ messages in thread
From: René Scharfe @ 2020-11-12 12:23 UTC (permalink / raw)
  To: Git Mailing List; +Cc: Junio C Hamano, Taylor Blau, Derrick Stolee

Call hashwrite_be64() to write a 64-bit value instead of open-coding it
using htonl() and hashwrite().  This shortens the code, gets rid of a
buffer and several magic numbers, and makes the intent clearer.

Signed-off-by: René Scharfe <l.s.r@web.de>
---
 pack-write.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/pack-write.c b/pack-write.c
index 23e19cc1ec..3513665e1e 100644
--- a/pack-write.c
+++ b/pack-write.c
@@ -151,13 +151,10 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		while (nr_large_offset) {
 			struct pack_idx_entry *obj = *list++;
 			uint64_t offset = obj->offset;
-			uint32_t split[2];

 			if (!need_large_offset(offset, opts))
 				continue;
-			split[0] = htonl(offset >> 32);
-			split[1] = htonl(offset & 0xffffffff);
-			hashwrite(f, split, 8);
+			hashwrite_be64(f, offset);
 			nr_large_offset--;
 		}
 	}
--
2.29.2

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] csum-file: add hashwrite_be64()
  2020-11-12 12:20 [PATCH 1/3] csum-file: add hashwrite_be64() René Scharfe
  2020-11-12 12:22 ` [PATCH 2/3] midx: use hashwrite_be64() René Scharfe
  2020-11-12 12:23 ` [PATCH 3/3] pack-write: " René Scharfe
@ 2020-11-12 13:52 ` Derrick Stolee
  2020-11-12 14:47   ` Taylor Blau
  2020-11-12 14:51 ` Taylor Blau
  3 siblings, 1 reply; 6+ messages in thread
From: Derrick Stolee @ 2020-11-12 13:52 UTC (permalink / raw)
  To: René Scharfe, Git Mailing List
  Cc: Junio C Hamano, Taylor Blau, Derrick Stolee

On 11/12/2020 7:20 AM, René Scharfe wrote:
> Add a helper function for hashing and writing 64-bit integers in network
> byte order.  It returns the number of written bytes.  This simplifies
> callers that keep track of the file offset, even though this number is a
> constant.
> 
> Suggested-by: Derrick Stolee <dstolee@microsoft.com>
> Original-patch-by: Taylor Blau <me@ttaylorr.com>

These patches are absolutely correct, and I'm glad to see them show up
in a very clear presentation. I had to go look to see why these were
not already present, with [1] being the last instance of these showing
up on-list. They did not get into the new version after a significant
refactor [2].

Thanks,
-Stolee

[1] https://lore.kernel.org/git/20200904202226.GA21837@nand.local/
[2] https://lore.kernel.org/git/cover.1599664389.git.me@ttaylorr.com/

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] csum-file: add hashwrite_be64()
  2020-11-12 13:52 ` [PATCH 1/3] csum-file: add hashwrite_be64() Derrick Stolee
@ 2020-11-12 14:47   ` Taylor Blau
  0 siblings, 0 replies; 6+ messages in thread
From: Taylor Blau @ 2020-11-12 14:47 UTC (permalink / raw)
  To: Derrick Stolee
  Cc: René Scharfe, Git Mailing List, Junio C Hamano, Taylor Blau,
	Derrick Stolee

On Thu, Nov 12, 2020 at 08:52:24AM -0500, Derrick Stolee wrote:
> On 11/12/2020 7:20 AM, René Scharfe wrote:
> > Add a helper function for hashing and writing 64-bit integers in network
> > byte order.  It returns the number of written bytes.  This simplifies
> > callers that keep track of the file offset, even though this number is a
> > constant.
> >
> > Suggested-by: Derrick Stolee <dstolee@microsoft.com>
> > Original-patch-by: Taylor Blau <me@ttaylorr.com>
>
> These patches are absolutely correct, and I'm glad to see them show up
> in a very clear presentation. I had to go look to see why these were
> not already present, with [1] being the last instance of these showing
> up on-list. They did not get into the new version after a significant
> refactor [2].

That's right. What happened was we stopped writing the uncompressed "has
an unpresentable Bloom filter" bitmap between the two versions you're
talking about. Since we wrote each word as a big-endian 8-byte unsigned
value, introducing hashwrite_be64() was useful in the earlier version.

But in the rewritten version, there were no _new_ callers that would
have wanted hashwrite_be64(), so I dropped those patches since the
series was already large.

That all being said, these are definitely useful patches to have, so I'm
glad to see them being dug back up. Thanks, René :-).

> Thanks,
> -Stolee
>
> [1] https://lore.kernel.org/git/20200904202226.GA21837@nand.local/
> [2] https://lore.kernel.org/git/cover.1599664389.git.me@ttaylorr.com/

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] csum-file: add hashwrite_be64()
  2020-11-12 12:20 [PATCH 1/3] csum-file: add hashwrite_be64() René Scharfe
                   ` (2 preceding siblings ...)
  2020-11-12 13:52 ` [PATCH 1/3] csum-file: add hashwrite_be64() Derrick Stolee
@ 2020-11-12 14:51 ` Taylor Blau
  3 siblings, 0 replies; 6+ messages in thread
From: Taylor Blau @ 2020-11-12 14:51 UTC (permalink / raw)
  To: René Scharfe; +Cc: Git Mailing List, Junio C Hamano, Derrick Stolee

On Thu, Nov 12, 2020 at 01:20:19PM +0100, René Scharfe wrote:
> Add a helper function for hashing and writing 64-bit integers in network
> byte order.  It returns the number of written bytes.  This simplifies
> callers that keep track of the file offset, even though this number is a
> constant.
>
> Suggested-by: Derrick Stolee <dstolee@microsoft.com>
> Original-patch-by: Taylor Blau <me@ttaylorr.com>
> Signed-off-by: René Scharfe <l.s.r@web.de>
> ---
>  csum-file.h | 7 +++++++
>  1 file changed, 7 insertions(+)
>
> diff --git a/csum-file.h b/csum-file.h
> index f9cbd317fb..e54d53d1d0 100644
> --- a/csum-file.h
> +++ b/csum-file.h
> @@ -62,4 +62,11 @@ static inline void hashwrite_be32(struct hashfile *f, uint32_t data)
>  	hashwrite(f, &data, sizeof(data));
>  }
>
> +static inline size_t hashwrite_be64(struct hashfile *f, uint64_t data)
> +{
> +	data = htonll(data);

Great. This is new from my patch (which wrote the high- and low four
bytes with two separate hashwrite_be32()'s), but I think it's a clear
improvement.

In addition to being more readable, we can use the bswap instruction
once instead of twice if it exists.

Please feel free to add my:

  Signed-off-by: Taylor Blau <me@ttaylorr.com>

Thanks,
Taylor

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-11-12 14:51 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-12 12:20 [PATCH 1/3] csum-file: add hashwrite_be64() René Scharfe
2020-11-12 12:22 ` [PATCH 2/3] midx: use hashwrite_be64() René Scharfe
2020-11-12 12:23 ` [PATCH 3/3] pack-write: " René Scharfe
2020-11-12 13:52 ` [PATCH 1/3] csum-file: add hashwrite_be64() Derrick Stolee
2020-11-12 14:47   ` Taylor Blau
2020-11-12 14:51 ` Taylor Blau

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git