[PATCH 00/12] object_id part 11 (the_hash_algo)

[PATCH 00/12] object_id part 11 (the_hash_algo)

[brian m. carlson]

This series includes various changes to adopt the use of the_hash_algo
for abstracting hash algorithms away.

The series moves much of the hash-related code to hash.h from cache.h,
drops the ctxsz member in favor of allowing stack-allocated hash
contexts, and switches object-related code to use the_hash_algo for
hashing.

Note that not all instances of calls to git_SHA1_* have been converted.
The diff line code, the push cert code, and patch IDs all have been left
alone for the moment because they are not related to object handling.
Pack checksums, on the other hand, have been converted.

The series is based off master, and has one minor conflict with Patryk
Obara's recent object_id series.

I will also be sending out preliminary test patches on top of this
series that wire up an alternate hash algorithm so that we can see what
tests break as a result.  (Hint: there's a lot of them.)

brian m. carlson (12):
  hash: move SHA-1 macros to hash.h
  hash: create union for hash context allocation
  builtin/index-pack: improve hash function abstraction
  builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo
  sha1_file: switch uses of SHA-1 to the_hash_algo
  fast-import: switch various uses of SHA-1 to the_hash_algo
  pack-check: convert various uses of SHA-1 to abstract forms
  pack-write: switch various SHA-1 values to abstract forms
  read-cache: abstract away uses of SHA-1
  csum-file: rename sha1file to hashfile
  csum-file: abstract uses of SHA-1
  bulk-checkin: abstract SHA-1 usage

 builtin/index-pack.c     | 108 +++++++++++++++++++++++------------------------
 builtin/pack-objects.c   |  52 +++++++++++------------
 builtin/unpack-objects.c |  18 ++++----
 bulk-checkin.c           |  28 ++++++------
 cache.h                  |  25 -----------
 csum-file.c              |  46 ++++++++++----------
 csum-file.h              |  38 ++++++++---------
 fast-import.c            |  68 ++++++++++++++---------------
 hash.h                   |  34 +++++++++++++--
 pack-bitmap-write.c      |  30 ++++++-------
 pack-check.c             |  32 +++++++-------
 pack-write.c             |  77 ++++++++++++++++-----------------
 pack.h                   |   4 +-
 read-cache.c             |  54 ++++++++++++------------
 sha1_file.c              |  54 ++++++++++++------------
 15 files changed, 335 insertions(+), 333 deletions(-)

[PATCH 01/12] hash: move SHA-1 macros to hash.h

[brian m. carlson]

Most of the other code dealing with SHA-1 and other hashes is located in
hash.h, which is in turn loaded by cache.h.  Move the SHA-1 macros to
hash.h as well, so we can use them in additional hash-related items in
the future.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 cache.h | 25 -------------------------
 hash.h  | 25 +++++++++++++++++++++++++
 2 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/cache.h b/cache.h
index d8b975a571..bfde6f757a 100644
--- a/cache.h
+++ b/cache.h
@@ -16,31 +16,6 @@
 #include "sha1-array.h"
 #include "repository.h"
 
-#ifndef platform_SHA_CTX
-/*
- * platform's underlying implementation of SHA-1; could be OpenSSL,
- * blk_SHA, Apple CommonCrypto, etc...  Note that including
- * SHA1_HEADER may have already defined platform_SHA_CTX for our
- * own implementations like block-sha1 and ppc-sha1, so we list
- * the default for OpenSSL compatible SHA-1 implementations here.
- */
-#define platform_SHA_CTX	SHA_CTX
-#define platform_SHA1_Init	SHA1_Init
-#define platform_SHA1_Update	SHA1_Update
-#define platform_SHA1_Final    	SHA1_Final
-#endif
-
-#define git_SHA_CTX		platform_SHA_CTX
-#define git_SHA1_Init		platform_SHA1_Init
-#define git_SHA1_Update		platform_SHA1_Update
-#define git_SHA1_Final		platform_SHA1_Final
-
-#ifdef SHA1_MAX_BLOCK_SIZE
-#include "compat/sha1-chunked.h"
-#undef git_SHA1_Update
-#define git_SHA1_Update		git_SHA1_Update_Chunked
-#endif
-
 #include <zlib.h>
 typedef struct git_zstream {
 	z_stream z;
diff --git a/hash.h b/hash.h
index 7d7a864f5d..7122dea7b3 100644
--- a/hash.h
+++ b/hash.h
@@ -15,6 +15,31 @@
 #include "block-sha1/sha1.h"
 #endif
 
+#ifndef platform_SHA_CTX
+/*
+ * platform's underlying implementation of SHA-1; could be OpenSSL,
+ * blk_SHA, Apple CommonCrypto, etc...  Note that including
+ * SHA1_HEADER may have already defined platform_SHA_CTX for our
+ * own implementations like block-sha1 and ppc-sha1, so we list
+ * the default for OpenSSL compatible SHA-1 implementations here.
+ */
+#define platform_SHA_CTX	SHA_CTX
+#define platform_SHA1_Init	SHA1_Init
+#define platform_SHA1_Update	SHA1_Update
+#define platform_SHA1_Final    	SHA1_Final
+#endif
+
+#define git_SHA_CTX		platform_SHA_CTX
+#define git_SHA1_Init		platform_SHA1_Init
+#define git_SHA1_Update		platform_SHA1_Update
+#define git_SHA1_Final		platform_SHA1_Final
+
+#ifdef SHA1_MAX_BLOCK_SIZE
+#include "compat/sha1-chunked.h"
+#undef git_SHA1_Update
+#define git_SHA1_Update		git_SHA1_Update_Chunked
+#endif
+
 /*
  * Note that these constants are suitable for indexing the hash_algos array and
  * comparing against each other, but are otherwise arbitrary, so they should not

[PATCH 02/12] hash: create union for hash context allocation

[brian m. carlson]

In various parts of our code, we want to allocate a structure
representing the internal state of a hash algorithm.  The original
implementation of the hash algorithm abstraction assumed we would do
that using heap allocations, and added a context size element to struct
git_hash_algo.  However, most of the existing code uses stack
allocations and conversion would needlessly complicate various parts of
the code.  Add a union for the purpose of allocating hash contexts on
the stack and a typedef for ease of use.  Remove the ctxsz element for
struct git_hash_algo, which is no longer very useful.

This does mean that stack allocations will grow slightly as additional
hash functions are added, but this should not be a significant problem,
since we don't allocate many hash contexts.  The improved usability and
benefits from avoiding dynamic allocation outweigh this small downside.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 hash.h      | 9 ++++++---
 sha1_file.c | 2 --
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/hash.h b/hash.h
index 7122dea7b3..365846a6b5 100644
--- a/hash.h
+++ b/hash.h
@@ -55,6 +55,12 @@
 /* Number of algorithms supported (including unknown). */
 #define GIT_HASH_NALGOS (GIT_HASH_SHA1 + 1)
 
+/* A suitably aligned type for stack allocations of hash contexts. */
+union git_hash_ctx {
+	git_SHA_CTX sha1;
+};
+typedef union git_hash_ctx git_hash_ctx;
+
 typedef void (*git_hash_init_fn)(void *ctx);
 typedef void (*git_hash_update_fn)(void *ctx, const void *in, size_t len);
 typedef void (*git_hash_final_fn)(unsigned char *hash, void *ctx);
@@ -69,9 +75,6 @@ struct git_hash_algo {
 	/* A four-byte version identifier, used in pack indices. */
 	uint32_t format_id;
 
-	/* The size of a hash context (e.g. git_SHA_CTX). */
-	size_t ctxsz;
-
 	/* The length of the hash in binary. */
 	size_t rawsz;
 
diff --git a/sha1_file.c b/sha1_file.c
index 3da70ac650..e61d93a6e8 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -75,7 +75,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = {
 		0x00000000,
 		0,
 		0,
-		0,
 		git_hash_unknown_init,
 		git_hash_unknown_update,
 		git_hash_unknown_final,
@@ -86,7 +85,6 @@ const struct git_hash_algo hash_algos[GIT_HASH_NALGOS] = {
 		"sha-1",
 		/* "sha1", big-endian */
 		0x73686131,
-		sizeof(git_SHA_CTX),
 		GIT_SHA1_RAWSZ,
 		GIT_SHA1_HEXSZ,
 		git_hash_sha1_init,

Re: [PATCH 02/12] hash: create union for hash context allocation

[Patryk Obara]

On 28/01/2018 16:57, brian m. carlson wrote:
> In various parts of our code, we want to allocate a structure
> representing the internal state of a hash algorithm.  The original
> implementation of the hash algorithm abstraction assumed we would do
> that using heap allocations, and added a context size element to struct
> git_hash_algo.  However, most of the existing code uses stack
> allocations and conversion would needlessly complicate various parts of
> the code.  Add a union for the purpose of allocating hash contexts on
> the stack and a typedef for ease of use.  Remove the ctxsz element for
> struct git_hash_algo, which is no longer very useful.

Overall, I am OK with this approach (it's straightforward change and 
cleanest way to replace direct calls to git_SHA1_* functions), but just 
to play devil's advocate: OpenSSL decided to sway users into heap 
allocated contexts, citing binary compatibility issues if they change 
the size of context structure. [1]

I think we might need to revisit this design decision in future - 
perhaps as soon as we'll transition away from calling git_SHA1_* 
functions directly.

> +/* A suitably aligned type for stack allocations of hash contexts. */
> +union git_hash_ctx {
> +	git_SHA_CTX sha1;
> +};
> +typedef union git_hash_ctx git_hash_ctx;
> +
>   typedef void (*git_hash_init_fn)(void *ctx);
>   typedef void (*git_hash_update_fn)(void *ctx, const void *in, size_t len);
>   typedef void (*git_hash_final_fn)(unsigned char *hash, void *ctx);

I think it would be appropriate to replace "void *ctx" with 
"git_hash_ctx *ctx". This way we can avoid unnecessary casting in 
git_hash_sha1_* functions.

[1] https://wiki.openssl.org/index.php/Manual:EVP_DigestInit(3)#NOTES

-- 
| ← Ceci n'est pas une pipe
Patryk Obara

Re: [PATCH 02/12] hash: create union for hash context allocation

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2358 bytes --]

On Sun, Jan 28, 2018 at 08:57:21PM +0100, Patryk Obara wrote:
> On 28/01/2018 16:57, brian m. carlson wrote:
> > In various parts of our code, we want to allocate a structure
> > representing the internal state of a hash algorithm.  The original
> > implementation of the hash algorithm abstraction assumed we would do
> > that using heap allocations, and added a context size element to struct
> > git_hash_algo.  However, most of the existing code uses stack
> > allocations and conversion would needlessly complicate various parts of
> > the code.  Add a union for the purpose of allocating hash contexts on
> > the stack and a typedef for ease of use.  Remove the ctxsz element for
> > struct git_hash_algo, which is no longer very useful.
> 
> Overall, I am OK with this approach (it's straightforward change and
> cleanest way to replace direct calls to git_SHA1_* functions), but just to
> play devil's advocate: OpenSSL decided to sway users into heap allocated
> contexts, citing binary compatibility issues if they change the size of
> context structure. [1]
> 
> I think we might need to revisit this design decision in future - perhaps as
> soon as we'll transition away from calling git_SHA1_* functions directly.

The approach I took was to keep the code as similar as possible to
what's there already.  If our hash implementation wants to use pointers,
it's okay for it to define git_SHA1_CTX to a pointer type, and
everything should still work.  We treat the type as fully opaque anyway
(outside of the actual hash implementation).

> > +/* A suitably aligned type for stack allocations of hash contexts. */
> > +union git_hash_ctx {
> > +	git_SHA_CTX sha1;
> > +};
> > +typedef union git_hash_ctx git_hash_ctx;
> > +
> >   typedef void (*git_hash_init_fn)(void *ctx);
> >   typedef void (*git_hash_update_fn)(void *ctx, const void *in, size_t len);
> >   typedef void (*git_hash_final_fn)(unsigned char *hash, void *ctx);
> 
> I think it would be appropriate to replace "void *ctx" with "git_hash_ctx
> *ctx". This way we can avoid unnecessary casting in git_hash_sha1_*
> functions.

Yeah, that does make more sense.  I'll make that change.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]

[PATCH 03/12] builtin/index-pack: improve hash function abstraction

[brian m. carlson]

Convert several uses of unsigned char [20] to struct object_id and
convert various hard-coded constants and uses of SHA-1 functions to use
the_hash_algo.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/index-pack.c | 90 ++++++++++++++++++++++++++--------------------------
 1 file changed, 45 insertions(+), 45 deletions(-)

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 4c51aec81f..40c000aca8 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -91,7 +91,7 @@ static unsigned int input_offset, input_len;
 static off_t consumed_bytes;
 static off_t max_input_size;
 static unsigned deepest_delta;
-static git_SHA_CTX input_ctx;
+static git_hash_ctx input_ctx;
 static uint32_t input_crc32;
 static int input_fd, output_fd;
 static const char *curr_pack;
@@ -253,7 +253,7 @@ static void flush(void)
 	if (input_offset) {
 		if (output_fd >= 0)
 			write_or_die(output_fd, input_buffer, input_offset);
-		git_SHA1_Update(&input_ctx, input_buffer, input_offset);
+		the_hash_algo->update_fn(&input_ctx, input_buffer, input_offset);
 		memmove(input_buffer, input_buffer + input_offset, input_len);
 		input_offset = 0;
 	}
@@ -326,7 +326,7 @@ static const char *open_pack_file(const char *pack_name)
 		output_fd = -1;
 		nothread_data.pack_fd = input_fd;
 	}
-	git_SHA1_Init(&input_ctx);
+	the_hash_algo->init_fn(&input_ctx);
 	return pack_name;
 }
 
@@ -437,22 +437,22 @@ static int is_delta_type(enum object_type type)
 }
 
 static void *unpack_entry_data(off_t offset, unsigned long size,
-			       enum object_type type, unsigned char *sha1)
+			       enum object_type type, struct object_id *oid)
 {
 	static char fixed_buf[8192];
 	int status;
 	git_zstream stream;
 	void *buf;
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	char hdr[32];
 	int hdrlen;
 
 	if (!is_delta_type(type)) {
 		hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(type), size) + 1;
-		git_SHA1_Init(&c);
-		git_SHA1_Update(&c, hdr, hdrlen);
+		the_hash_algo->init_fn(&c);
+		the_hash_algo->update_fn(&c, hdr, hdrlen);
 	} else
-		sha1 = NULL;
+		oid = NULL;
 	if (type == OBJ_BLOB && size > big_file_threshold)
 		buf = fixed_buf;
 	else
@@ -469,8 +469,8 @@ static void *unpack_entry_data(off_t offset, unsigned long size,
 		stream.avail_in = input_len;
 		status = git_inflate(&stream, 0);
 		use(input_len - stream.avail_in);
-		if (sha1)
-			git_SHA1_Update(&c, last_out, stream.next_out - last_out);
+		if (oid)
+			the_hash_algo->update_fn(&c, last_out, stream.next_out - last_out);
 		if (buf == fixed_buf) {
 			stream.next_out = buf;
 			stream.avail_out = sizeof(fixed_buf);
@@ -479,15 +479,15 @@ static void *unpack_entry_data(off_t offset, unsigned long size,
 	if (stream.total_out != size || status != Z_STREAM_END)
 		bad_object(offset, _("inflate returned %d"), status);
 	git_inflate_end(&stream);
-	if (sha1)
-		git_SHA1_Final(sha1, &c);
+	if (oid)
+		the_hash_algo->final_fn(oid->hash, &c);
 	return buf == fixed_buf ? NULL : buf;
 }
 
 static void *unpack_raw_entry(struct object_entry *obj,
 			      off_t *ofs_offset,
-			      unsigned char *ref_sha1,
-			      unsigned char *sha1)
+			      struct object_id *ref_oid,
+			      struct object_id *oid)
 {
 	unsigned char *p;
 	unsigned long size, c;
@@ -515,8 +515,8 @@ static void *unpack_raw_entry(struct object_entry *obj,
 
 	switch (obj->type) {
 	case OBJ_REF_DELTA:
-		hashcpy(ref_sha1, fill(20));
-		use(20);
+		hashcpy(ref_oid->hash, fill(the_hash_algo->rawsz));
+		use(the_hash_algo->rawsz);
 		break;
 	case OBJ_OFS_DELTA:
 		p = fill(1);
@@ -546,7 +546,7 @@ static void *unpack_raw_entry(struct object_entry *obj,
 	}
 	obj->hdr_size = consumed_bytes - obj->idx.offset;
 
-	data = unpack_entry_data(obj->idx.offset, obj->size, obj->type, sha1);
+	data = unpack_entry_data(obj->idx.offset, obj->size, obj->type, oid);
 	obj->idx.crc32 = input_crc32;
 	return data;
 }
@@ -1119,11 +1119,11 @@ static void *threaded_second_pass(void *data)
  * - calculate SHA1 of all non-delta objects;
  * - remember base (SHA1 or offset) for all deltas.
  */
-static void parse_pack_objects(unsigned char *sha1)
+static void parse_pack_objects(unsigned char *hash)
 {
 	int i, nr_delays = 0;
 	struct ofs_delta_entry *ofs_delta = ofs_deltas;
-	unsigned char ref_delta_sha1[20];
+	struct object_id ref_delta_oid;
 	struct stat st;
 
 	if (verbose)
@@ -1133,8 +1133,8 @@ static void parse_pack_objects(unsigned char *sha1)
 	for (i = 0; i < nr_objects; i++) {
 		struct object_entry *obj = &objects[i];
 		void *data = unpack_raw_entry(obj, &ofs_delta->offset,
-					      ref_delta_sha1,
-					      obj->idx.oid.hash);
+					      &ref_delta_oid,
+					      &obj->idx.oid);
 		obj->real_type = obj->type;
 		if (obj->type == OBJ_OFS_DELTA) {
 			nr_ofs_deltas++;
@@ -1142,7 +1142,7 @@ static void parse_pack_objects(unsigned char *sha1)
 			ofs_delta++;
 		} else if (obj->type == OBJ_REF_DELTA) {
 			ALLOC_GROW(ref_deltas, nr_ref_deltas + 1, ref_deltas_alloc);
-			hashcpy(ref_deltas[nr_ref_deltas].sha1, ref_delta_sha1);
+			hashcpy(ref_deltas[nr_ref_deltas].sha1, ref_delta_oid.hash);
 			ref_deltas[nr_ref_deltas].obj_no = i;
 			nr_ref_deltas++;
 		} else if (!data) {
@@ -1160,10 +1160,10 @@ static void parse_pack_objects(unsigned char *sha1)
 
 	/* Check pack integrity */
 	flush();
-	git_SHA1_Final(sha1, &input_ctx);
-	if (hashcmp(fill(20), sha1))
+	the_hash_algo->final_fn(hash, &input_ctx);
+	if (hashcmp(fill(the_hash_algo->rawsz), hash))
 		die(_("pack is corrupted (SHA1 mismatch)"));
-	use(20);
+	use(the_hash_algo->rawsz);
 
 	/* If input_fd is a file, we should have reached its end now. */
 	if (fstat(input_fd, &st))
@@ -1239,21 +1239,21 @@ static void resolve_deltas(void)
 /*
  * Third pass:
  * - append objects to convert thin pack to full pack if required
- * - write the final 20-byte SHA-1
+ * - write the final pack hash
  */
 static void fix_unresolved_deltas(struct sha1file *f);
-static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_sha1)
+static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_hash)
 {
 	if (nr_ref_deltas + nr_ofs_deltas == nr_resolved_deltas) {
 		stop_progress(&progress);
-		/* Flush remaining pack final 20-byte SHA1. */
+		/* Flush remaining pack final hash. */
 		flush();
 		return;
 	}
 
 	if (fix_thin_pack) {
 		struct sha1file *f;
-		unsigned char read_sha1[20], tail_sha1[20];
+		unsigned char read_hash[GIT_MAX_RAWSZ], tail_hash[GIT_MAX_RAWSZ];
 		struct strbuf msg = STRBUF_INIT;
 		int nr_unresolved = nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas;
 		int nr_objects_initial = nr_objects;
@@ -1270,12 +1270,12 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 			    nr_objects - nr_objects_initial);
 		stop_progress_msg(&progress, msg.buf);
 		strbuf_release(&msg);
-		sha1close(f, tail_sha1, 0);
-		hashcpy(read_sha1, pack_sha1);
-		fixup_pack_header_footer(output_fd, pack_sha1,
+		sha1close(f, tail_hash, 0);
+		hashcpy(read_hash, pack_hash);
+		fixup_pack_header_footer(output_fd, pack_hash,
 					 curr_pack, nr_objects,
-					 read_sha1, consumed_bytes-20);
-		if (hashcmp(read_sha1, tail_sha1) != 0)
+					 read_hash, consumed_bytes-the_hash_algo->rawsz);
+		if (hashcmp(read_hash, tail_hash) != 0)
 			die(_("Unexpected tail checksum for %s "
 			      "(disk corruption?)"), curr_pack);
 	}
@@ -1392,7 +1392,7 @@ static void fix_unresolved_deltas(struct sha1file *f)
 static void final(const char *final_pack_name, const char *curr_pack_name,
 		  const char *final_index_name, const char *curr_index_name,
 		  const char *keep_name, const char *keep_msg,
-		  unsigned char *sha1)
+		  unsigned char *hash)
 {
 	const char *report = "pack";
 	struct strbuf pack_name = STRBUF_INIT;
@@ -1413,7 +1413,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 		int keep_fd, keep_msg_len = strlen(keep_msg);
 
 		if (!keep_name)
-			keep_name = odb_pack_name(&keep_name_buf, sha1, "keep");
+			keep_name = odb_pack_name(&keep_name_buf, hash, "keep");
 
 		keep_fd = odb_pack_keep(keep_name);
 		if (keep_fd < 0) {
@@ -1434,7 +1434,7 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 
 	if (final_pack_name != curr_pack_name) {
 		if (!final_pack_name)
-			final_pack_name = odb_pack_name(&pack_name, sha1, "pack");
+			final_pack_name = odb_pack_name(&pack_name, hash, "pack");
 		if (finalize_object_file(curr_pack_name, final_pack_name))
 			die(_("cannot store pack file"));
 	} else if (from_stdin)
@@ -1442,18 +1442,18 @@ static void final(const char *final_pack_name, const char *curr_pack_name,
 
 	if (final_index_name != curr_index_name) {
 		if (!final_index_name)
-			final_index_name = odb_pack_name(&index_name, sha1, "idx");
+			final_index_name = odb_pack_name(&index_name, hash, "idx");
 		if (finalize_object_file(curr_index_name, final_index_name))
 			die(_("cannot store index file"));
 	} else
 		chmod(final_index_name, 0444);
 
 	if (!from_stdin) {
-		printf("%s\n", sha1_to_hex(sha1));
+		printf("%s\n", sha1_to_hex(hash));
 	} else {
 		struct strbuf buf = STRBUF_INIT;
 
-		strbuf_addf(&buf, "%s\t%s\n", report, sha1_to_hex(sha1));
+		strbuf_addf(&buf, "%s\t%s\n", report, sha1_to_hex(hash));
 		write_or_die(1, buf.buf, buf.len);
 		strbuf_release(&buf);
 
@@ -1637,7 +1637,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
 		      keep_name_buf = STRBUF_INIT;
 	struct pack_idx_entry **idx_objects;
 	struct pack_idx_option opts;
-	unsigned char pack_sha1[20];
+	unsigned char pack_hash[GIT_MAX_RAWSZ];
 	unsigned foreign_nr = 1;	/* zero is a "good" value, assume bad */
 	int report_end_of_input = 0;
 
@@ -1768,11 +1768,11 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
 	if (show_stat)
 		obj_stat = xcalloc(st_add(nr_objects, 1), sizeof(struct object_stat));
 	ofs_deltas = xcalloc(nr_objects, sizeof(struct ofs_delta_entry));
-	parse_pack_objects(pack_sha1);
+	parse_pack_objects(pack_hash);
 	if (report_end_of_input)
 		write_in_full(2, "\0", 1);
 	resolve_deltas();
-	conclude_pack(fix_thin_pack, curr_pack, pack_sha1);
+	conclude_pack(fix_thin_pack, curr_pack, pack_hash);
 	free(ofs_deltas);
 	free(ref_deltas);
 	if (strict)
@@ -1784,14 +1784,14 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
 	ALLOC_ARRAY(idx_objects, nr_objects);
 	for (i = 0; i < nr_objects; i++)
 		idx_objects[i] = &objects[i].idx;
-	curr_index = write_idx_file(index_name, idx_objects, nr_objects, &opts, pack_sha1);
+	curr_index = write_idx_file(index_name, idx_objects, nr_objects, &opts, pack_hash);
 	free(idx_objects);
 
 	if (!verify)
 		final(pack_name, curr_pack,
 		      index_name, curr_index,
 		      keep_name, keep_msg,
-		      pack_sha1);
+		      pack_hash);
 	else
 		close(input_fd);
 	free(objects);

[PATCH 04/12] builtin/unpack-objects: switch uses of SHA-1 to the_hash_algo

[brian m. carlson]

Switch various uses of explicit calls to SHA-1 into references to
the_hash_algo to better abstract away the various uses of it.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/unpack-objects.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c
index 62ea264c46..813ca31979 100644
--- a/builtin/unpack-objects.c
+++ b/builtin/unpack-objects.c
@@ -21,7 +21,7 @@ static unsigned char buffer[4096];
 static unsigned int offset, len;
 static off_t consumed_bytes;
 static off_t max_input_size;
-static git_SHA_CTX ctx;
+static git_hash_ctx ctx;
 static struct fsck_options fsck_options = FSCK_OPTIONS_STRICT;
 
 /*
@@ -62,7 +62,7 @@ static void *fill(int min)
 	if (min > sizeof(buffer))
 		die("cannot fill %d bytes", min);
 	if (offset) {
-		git_SHA1_Update(&ctx, buffer, offset);
+		the_hash_algo->update_fn(&ctx, buffer, offset);
 		memmove(buffer, buffer + offset, len);
 		offset = 0;
 	}
@@ -345,8 +345,8 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size,
 	struct object_id base_oid;
 
 	if (type == OBJ_REF_DELTA) {
-		hashcpy(base_oid.hash, fill(GIT_SHA1_RAWSZ));
-		use(GIT_SHA1_RAWSZ);
+		hashcpy(base_oid.hash, fill(the_hash_algo->rawsz));
+		use(the_hash_algo->rawsz);
 		delta_data = get_data(delta_size);
 		if (dry_run || !delta_data) {
 			free(delta_data);
@@ -564,15 +564,15 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix)
 		/* We don't take any non-flag arguments now.. Maybe some day */
 		usage(unpack_usage);
 	}
-	git_SHA1_Init(&ctx);
+	the_hash_algo->init_fn(&ctx);
 	unpack_all();
-	git_SHA1_Update(&ctx, buffer, offset);
-	git_SHA1_Final(oid.hash, &ctx);
+	the_hash_algo->update_fn(&ctx, buffer, offset);
+	the_hash_algo->final_fn(oid.hash, &ctx);
 	if (strict)
 		write_rest();
-	if (hashcmp(fill(GIT_SHA1_RAWSZ), oid.hash))
+	if (hashcmp(fill(the_hash_algo->rawsz), oid.hash))
 		die("final sha1 did not match");
-	use(GIT_SHA1_RAWSZ);
+	use(the_hash_algo->rawsz);
 
 	/* Write the last part of the buffer to stdout */
 	while (len) {

[PATCH 05/12] sha1_file: switch uses of SHA-1 to the_hash_algo

[brian m. carlson]

Switch various uses of explicit calls to SHA-1 into references to
the_hash_algo for better abstraction.  Convert some calls to use struct
object_id.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 sha1_file.c | 52 ++++++++++++++++++++++++++--------------------------
 1 file changed, 26 insertions(+), 26 deletions(-)

diff --git a/sha1_file.c b/sha1_file.c
index e61d93a6e8..d9e2b1f285 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -786,16 +786,16 @@ void *xmmap(void *start, size_t length,
 int check_sha1_signature(const unsigned char *sha1, void *map,
 			 unsigned long size, const char *type)
 {
-	unsigned char real_sha1[20];
+	struct object_id real_oid;
 	enum object_type obj_type;
 	struct git_istream *st;
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	char hdr[32];
 	int hdrlen;
 
 	if (map) {
-		hash_sha1_file(map, size, type, real_sha1);
-		return hashcmp(sha1, real_sha1) ? -1 : 0;
+		hash_sha1_file(map, size, type, real_oid.hash);
+		return hashcmp(sha1, real_oid.hash) ? -1 : 0;
 	}
 
 	st = open_istream(sha1, &obj_type, &size, NULL);
@@ -806,8 +806,8 @@ int check_sha1_signature(const unsigned char *sha1, void *map,
 	hdrlen = xsnprintf(hdr, sizeof(hdr), "%s %lu", typename(obj_type), size) + 1;
 
 	/* Sha1.. */
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, hdr, hdrlen);
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, hdr, hdrlen);
 	for (;;) {
 		char buf[1024 * 16];
 		ssize_t readlen = read_istream(st, buf, sizeof(buf));
@@ -818,11 +818,11 @@ int check_sha1_signature(const unsigned char *sha1, void *map,
 		}
 		if (!readlen)
 			break;
-		git_SHA1_Update(&c, buf, readlen);
+		the_hash_algo->update_fn(&c, buf, readlen);
 	}
-	git_SHA1_Final(real_sha1, &c);
+	the_hash_algo->final_fn(real_oid.hash, &c);
 	close_istream(st);
-	return hashcmp(sha1, real_sha1) ? -1 : 0;
+	return hashcmp(sha1, real_oid.hash) ? -1 : 0;
 }
 
 int git_open_cloexec(const char *name, int flags)
@@ -1421,16 +1421,16 @@ static void write_sha1_file_prepare(const void *buf, unsigned long len,
                                     const char *type, unsigned char *sha1,
                                     char *hdr, int *hdrlen)
 {
-	git_SHA_CTX c;
+	git_hash_ctx c;
 
 	/* Generate the header */
 	*hdrlen = xsnprintf(hdr, *hdrlen, "%s %lu", type, len)+1;
 
 	/* Sha1.. */
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, hdr, *hdrlen);
-	git_SHA1_Update(&c, buf, len);
-	git_SHA1_Final(sha1, &c);
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, hdr, *hdrlen);
+	the_hash_algo->update_fn(&c, buf, len);
+	the_hash_algo->final_fn(sha1, &c);
 }
 
 /*
@@ -1552,8 +1552,8 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	int fd, ret;
 	unsigned char compressed[4096];
 	git_zstream stream;
-	git_SHA_CTX c;
-	unsigned char parano_sha1[20];
+	git_hash_ctx c;
+	struct object_id parano_oid;
 	static struct strbuf tmp_file = STRBUF_INIT;
 	const char *filename = sha1_file_name(sha1);
 
@@ -1569,14 +1569,14 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	git_deflate_init(&stream, zlib_compression_level);
 	stream.next_out = compressed;
 	stream.avail_out = sizeof(compressed);
-	git_SHA1_Init(&c);
+	the_hash_algo->init_fn(&c);
 
 	/* First header.. */
 	stream.next_in = (unsigned char *)hdr;
 	stream.avail_in = hdrlen;
 	while (git_deflate(&stream, 0) == Z_OK)
 		; /* nothing */
-	git_SHA1_Update(&c, hdr, hdrlen);
+	the_hash_algo->update_fn(&c, hdr, hdrlen);
 
 	/* Then the data itself.. */
 	stream.next_in = (void *)buf;
@@ -1584,7 +1584,7 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	do {
 		unsigned char *in0 = stream.next_in;
 		ret = git_deflate(&stream, Z_FINISH);
-		git_SHA1_Update(&c, in0, stream.next_in - in0);
+		the_hash_algo->update_fn(&c, in0, stream.next_in - in0);
 		if (write_buffer(fd, compressed, stream.next_out - compressed) < 0)
 			die("unable to write sha1 file");
 		stream.next_out = compressed;
@@ -1596,8 +1596,8 @@ static int write_loose_object(const unsigned char *sha1, char *hdr, int hdrlen,
 	ret = git_deflate_end_gently(&stream);
 	if (ret != Z_OK)
 		die("deflateEnd on object %s failed (%d)", sha1_to_hex(sha1), ret);
-	git_SHA1_Final(parano_sha1, &c);
-	if (hashcmp(sha1, parano_sha1) != 0)
+	the_hash_algo->final_fn(parano_oid.hash, &c);
+	if (hashcmp(sha1, parano_oid.hash) != 0)
 		die("confused by unstable object source data for %s", sha1_to_hex(sha1));
 
 	close_sha1_file(fd);
@@ -2091,14 +2091,14 @@ static int check_stream_sha1(git_zstream *stream,
 			     const char *path,
 			     const unsigned char *expected_sha1)
 {
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	unsigned char real_sha1[GIT_MAX_RAWSZ];
 	unsigned char buf[4096];
 	unsigned long total_read;
 	int status = Z_OK;
 
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, hdr, stream->total_out);
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, hdr, stream->total_out);
 
 	/*
 	 * We already read some bytes into hdr, but the ones up to the NUL
@@ -2117,7 +2117,7 @@ static int check_stream_sha1(git_zstream *stream,
 		if (size - total_read < stream->avail_out)
 			stream->avail_out = size - total_read;
 		status = git_inflate(stream, Z_FINISH);
-		git_SHA1_Update(&c, buf, stream->next_out - buf);
+		the_hash_algo->update_fn(&c, buf, stream->next_out - buf);
 		total_read += stream->next_out - buf;
 	}
 	git_inflate_end(stream);
@@ -2132,7 +2132,7 @@ static int check_stream_sha1(git_zstream *stream,
 		return -1;
 	}
 
-	git_SHA1_Final(real_sha1, &c);
+	the_hash_algo->final_fn(real_sha1, &c);
 	if (hashcmp(expected_sha1, real_sha1)) {
 		error("sha1 mismatch for %s (expected %s)", path,
 		      sha1_to_hex(expected_sha1));

[PATCH 06/12] fast-import: switch various uses of SHA-1 to the_hash_algo

[brian m. carlson]

Switch various uses of explicit calls to SHA-1 to use the_hash_algo.
Convert various uses of 20 and the GIT_SHA1 constants as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 fast-import.c | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index b70ac025e0..1b8ab8ea29 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1092,15 +1092,15 @@ static int store_object(
 	unsigned char hdr[96];
 	struct object_id oid;
 	unsigned long hdrlen, deltalen;
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	git_zstream s;
 
 	hdrlen = xsnprintf((char *)hdr, sizeof(hdr), "%s %lu",
 			   typename(type), (unsigned long)dat->len) + 1;
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, hdr, hdrlen);
-	git_SHA1_Update(&c, dat->buf, dat->len);
-	git_SHA1_Final(oid.hash, &c);
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, hdr, hdrlen);
+	the_hash_algo->update_fn(&c, dat->buf, dat->len);
+	the_hash_algo->final_fn(oid.hash, &c);
 	if (oidout)
 		oidcpy(oidout, &oid);
 
@@ -1118,11 +1118,11 @@ static int store_object(
 		return 1;
 	}
 
-	if (last && last->data.buf && last->depth < max_depth && dat->len > 20) {
+	if (last && last->data.buf && last->depth < max_depth && dat->len > the_hash_algo->rawsz) {
 		delta_count_attempts_by_type[type]++;
 		delta = diff_delta(last->data.buf, last->data.len,
 			dat->buf, dat->len,
-			&deltalen, dat->len - 20);
+			&deltalen, dat->len - the_hash_algo->rawsz);
 	} else
 		delta = NULL;
 
@@ -1231,7 +1231,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 	struct object_id oid;
 	unsigned long hdrlen;
 	off_t offset;
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	git_zstream s;
 	struct sha1file_checkpoint checkpoint;
 	int status = Z_OK;
@@ -1246,8 +1246,8 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 
 	hdrlen = xsnprintf((char *)out_buf, out_sz, "blob %" PRIuMAX, len) + 1;
 
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, out_buf, hdrlen);
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, out_buf, hdrlen);
 
 	crc32_begin(pack_file);
 
@@ -1265,7 +1265,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 			if (!n && feof(stdin))
 				die("EOF in data (%" PRIuMAX " bytes remaining)", len);
 
-			git_SHA1_Update(&c, in_buf, n);
+			the_hash_algo->update_fn(&c, in_buf, n);
 			s.next_in = in_buf;
 			s.avail_in = n;
 			len -= n;
@@ -1291,7 +1291,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		}
 	}
 	git_deflate_end(&s);
-	git_SHA1_Final(oid.hash, &c);
+	the_hash_algo->final_fn(oid.hash, &c);
 
 	if (oidout)
 		oidcpy(oidout, &oid);
@@ -1350,11 +1350,11 @@ static void *gfi_unpack_entry(
 {
 	enum object_type type;
 	struct packed_git *p = all_packs[oe->pack_id];
-	if (p == pack_data && p->pack_size < (pack_size + 20)) {
+	if (p == pack_data && p->pack_size < (pack_size + the_hash_algo->rawsz)) {
 		/* The object is stored in the packfile we are writing to
 		 * and we have modified it since the last time we scanned
 		 * back to read a previously written object.  If an old
-		 * window covered [p->pack_size, p->pack_size + 20) its
+		 * window covered [p->pack_size, p->pack_size + rawsz) its
 		 * data is stale and is not valid.  Closing all windows
 		 * and updating the packfile length ensures we can read
 		 * the newly written data.
@@ -1362,13 +1362,13 @@ static void *gfi_unpack_entry(
 		close_pack_windows(p);
 		sha1flush(pack_file);
 
-		/* We have to offer 20 bytes additional on the end of
+		/* We have to offer rawsz bytes additional on the end of
 		 * the packfile as the core unpacker code assumes the
 		 * footer is present at the file end and must promise
-		 * at least 20 bytes within any window it maps.  But
+		 * at least rawsz bytes within any window it maps.  But
 		 * we don't actually create the footer here.
 		 */
-		p->pack_size = pack_size + 20;
+		p->pack_size = pack_size + the_hash_algo->rawsz;
 	}
 	return unpack_entry(p, oe->idx.offset, &type, sizep);
 }
@@ -2204,7 +2204,7 @@ static void construct_path_with_fanout(const char *hex_sha1,
 		unsigned char fanout, char *path)
 {
 	unsigned int i = 0, j = 0;
-	if (fanout >= 20)
+	if (fanout >= the_hash_algo->rawsz)
 		die("Too large fanout (%u)", fanout);
 	while (fanout) {
 		path[i++] = hex_sha1[j++];
@@ -2212,8 +2212,8 @@ static void construct_path_with_fanout(const char *hex_sha1,
 		path[i++] = '/';
 		fanout--;
 	}
-	memcpy(path + i, hex_sha1 + j, GIT_SHA1_HEXSZ - j);
-	path[i + GIT_SHA1_HEXSZ - j] = '\0';
+	memcpy(path + i, hex_sha1 + j, the_hash_algo->hexsz - j);
+	path[i + the_hash_algo->hexsz - j] = '\0';
 }
 
 static uintmax_t do_change_note_fanout(

Re: [PATCH 06/12] fast-import: switch various uses of SHA-1 to the_hash_algo

[Patryk Obara]

On 28/01/2018 16:57, brian m. carlson wrote:
> -	if (last && last->data.buf && last->depth < max_depth && dat->len > 20) {
> +	if (last && last->data.buf && last->depth < max_depth && dat->len > the_hash_algo->rawsz) {

At this point line is almost 100 characters long - maybe it's time to
break it ;)

-- 
| ← Ceci n'est pas une pipe
Patryk Obara

[PATCH 07/12] pack-check: convert various uses of SHA-1 to abstract forms

[brian m. carlson]

Convert various explicit calls to use SHA-1 functions and constants to
references to the_hash_algo.  Make several strings more generic with
respect to the hash algorithm used.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 pack-check.c | 32 ++++++++++++++++----------------
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/pack-check.c b/pack-check.c
index 073c1fbd46..403a572567 100644
--- a/pack-check.c
+++ b/pack-check.c
@@ -41,7 +41,7 @@ int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,
 	} while (len);
 
 	index_crc = p->index_data;
-	index_crc += 2 + 256 + p->num_objects * (20/4) + nr;
+	index_crc += 2 + 256 + p->num_objects * (the_hash_algo->rawsz/4) + nr;
 
 	return data_crc != ntohl(*index_crc);
 }
@@ -54,7 +54,7 @@ static int verify_packfile(struct packed_git *p,
 {
 	off_t index_size = p->index_size;
 	const unsigned char *index_base = p->index_data;
-	git_SHA_CTX ctx;
+	git_hash_ctx ctx;
 	unsigned char hash[GIT_MAX_RAWSZ], *pack_sig;
 	off_t offset = 0, pack_sig_ofs = 0;
 	uint32_t nr_objects, i;
@@ -64,24 +64,24 @@ static int verify_packfile(struct packed_git *p,
 	if (!is_pack_valid(p))
 		return error("packfile %s cannot be accessed", p->pack_name);
 
-	git_SHA1_Init(&ctx);
+	the_hash_algo->init_fn(&ctx);
 	do {
 		unsigned long remaining;
 		unsigned char *in = use_pack(p, w_curs, offset, &remaining);
 		offset += remaining;
 		if (!pack_sig_ofs)
-			pack_sig_ofs = p->pack_size - 20;
+			pack_sig_ofs = p->pack_size - the_hash_algo->rawsz;
 		if (offset > pack_sig_ofs)
 			remaining -= (unsigned int)(offset - pack_sig_ofs);
-		git_SHA1_Update(&ctx, in, remaining);
+		the_hash_algo->update_fn(&ctx, in, remaining);
 	} while (offset < pack_sig_ofs);
-	git_SHA1_Final(hash, &ctx);
+	the_hash_algo->final_fn(hash, &ctx);
 	pack_sig = use_pack(p, w_curs, pack_sig_ofs, NULL);
 	if (hashcmp(hash, pack_sig))
-		err = error("%s SHA1 checksum mismatch",
+		err = error("%s pack checksum mismatch",
 			    p->pack_name);
-	if (hashcmp(index_base + index_size - 40, pack_sig))
-		err = error("%s SHA1 does not match its index",
+	if (hashcmp(index_base + index_size - the_hash_algo->hexsz, pack_sig))
+		err = error("%s pack checksum does not match its index",
 			    p->pack_name);
 	unuse_pack(w_curs);
 
@@ -165,8 +165,8 @@ int verify_pack_index(struct packed_git *p)
 {
 	off_t index_size;
 	const unsigned char *index_base;
-	git_SHA_CTX ctx;
-	unsigned char sha1[20];
+	git_hash_ctx ctx;
+	unsigned char hash[GIT_MAX_RAWSZ];
 	int err = 0;
 
 	if (open_pack_index(p))
@@ -175,11 +175,11 @@ int verify_pack_index(struct packed_git *p)
 	index_base = p->index_data;
 
 	/* Verify SHA1 sum of the index file */
-	git_SHA1_Init(&ctx);
-	git_SHA1_Update(&ctx, index_base, (unsigned int)(index_size - 20));
-	git_SHA1_Final(sha1, &ctx);
-	if (hashcmp(sha1, index_base + index_size - 20))
-		err = error("Packfile index for %s SHA1 mismatch",
+	the_hash_algo->init_fn(&ctx);
+	the_hash_algo->update_fn(&ctx, index_base, (unsigned int)(index_size - the_hash_algo->rawsz));
+	the_hash_algo->final_fn(hash, &ctx);
+	if (hashcmp(hash, index_base + index_size - the_hash_algo->rawsz))
+		err = error("Packfile index for %s hash mismatch",
 			    p->pack_name);
 	return err;
 }

[PATCH 08/12] pack-write: switch various SHA-1 values to abstract forms

[brian m. carlson]

Convert various uses of hardcoded 20- and 40-based numbers to use
the_hash_algo, along with direct calls to SHA-1.  Adjust the names of
variables to refer to "hash" instead of "sha1".

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 pack-write.c | 49 +++++++++++++++++++++++++------------------------
 1 file changed, 25 insertions(+), 24 deletions(-)

diff --git a/pack-write.c b/pack-write.c
index fea6284192..fe33f7464c 100644
--- a/pack-write.c
+++ b/pack-write.c
@@ -122,7 +122,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 			uint32_t offset = htonl(obj->offset);
 			sha1write(f, &offset, 4);
 		}
-		sha1write(f, obj->oid.hash, 20);
+		sha1write(f, obj->oid.hash, the_hash_algo->rawsz);
 		if ((opts->flags & WRITE_IDX_STRICT) &&
 		    (i && !oidcmp(&list[-2]->oid, &obj->oid)))
 			die("The same object %s appears twice in the pack",
@@ -169,7 +169,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		}
 	}
 
-	sha1write(f, sha1, 20);
+	sha1write(f, sha1, the_hash_algo->rawsz);
 	sha1close(f, NULL, ((opts->flags & WRITE_IDX_VERIFY)
 			    ? CSUM_CLOSE : CSUM_FSYNC));
 	return index_name;
@@ -203,20 +203,20 @@ off_t write_pack_header(struct sha1file *f, uint32_t nr_entries)
  * interested in the resulting SHA1 of pack data above partial_pack_offset.
  */
 void fixup_pack_header_footer(int pack_fd,
-			 unsigned char *new_pack_sha1,
+			 unsigned char *new_pack_hash,
 			 const char *pack_name,
 			 uint32_t object_count,
-			 unsigned char *partial_pack_sha1,
+			 unsigned char *partial_pack_hash,
 			 off_t partial_pack_offset)
 {
 	int aligned_sz, buf_sz = 8 * 1024;
-	git_SHA_CTX old_sha1_ctx, new_sha1_ctx;
+	git_hash_ctx old_hash_ctx, new_hash_ctx;
 	struct pack_header hdr;
 	char *buf;
 	ssize_t read_result;
 
-	git_SHA1_Init(&old_sha1_ctx);
-	git_SHA1_Init(&new_sha1_ctx);
+	the_hash_algo->init_fn(&old_hash_ctx);
+	the_hash_algo->init_fn(&new_hash_ctx);
 
 	if (lseek(pack_fd, 0, SEEK_SET) != 0)
 		die_errno("Failed seeking to start of '%s'", pack_name);
@@ -228,9 +228,9 @@ void fixup_pack_header_footer(int pack_fd,
 			  pack_name);
 	if (lseek(pack_fd, 0, SEEK_SET) != 0)
 		die_errno("Failed seeking to start of '%s'", pack_name);
-	git_SHA1_Update(&old_sha1_ctx, &hdr, sizeof(hdr));
+	the_hash_algo->update_fn(&old_hash_ctx, &hdr, sizeof(hdr));
 	hdr.hdr_entries = htonl(object_count);
-	git_SHA1_Update(&new_sha1_ctx, &hdr, sizeof(hdr));
+	the_hash_algo->update_fn(&new_hash_ctx, &hdr, sizeof(hdr));
 	write_or_die(pack_fd, &hdr, sizeof(hdr));
 	partial_pack_offset -= sizeof(hdr);
 
@@ -238,28 +238,28 @@ void fixup_pack_header_footer(int pack_fd,
 	aligned_sz = buf_sz - sizeof(hdr);
 	for (;;) {
 		ssize_t m, n;
-		m = (partial_pack_sha1 && partial_pack_offset < aligned_sz) ?
+		m = (partial_pack_hash && partial_pack_offset < aligned_sz) ?
 			partial_pack_offset : aligned_sz;
 		n = xread(pack_fd, buf, m);
 		if (!n)
 			break;
 		if (n < 0)
 			die_errno("Failed to checksum '%s'", pack_name);
-		git_SHA1_Update(&new_sha1_ctx, buf, n);
+		the_hash_algo->update_fn(&new_hash_ctx, buf, n);
 
 		aligned_sz -= n;
 		if (!aligned_sz)
 			aligned_sz = buf_sz;
 
-		if (!partial_pack_sha1)
+		if (!partial_pack_hash)
 			continue;
 
-		git_SHA1_Update(&old_sha1_ctx, buf, n);
+		the_hash_algo->update_fn(&old_hash_ctx, buf, n);
 		partial_pack_offset -= n;
 		if (partial_pack_offset == 0) {
-			unsigned char sha1[20];
-			git_SHA1_Final(sha1, &old_sha1_ctx);
-			if (hashcmp(sha1, partial_pack_sha1) != 0)
+			unsigned char hash[GIT_MAX_RAWSZ];
+			the_hash_algo->final_fn(hash, &old_hash_ctx);
+			if (hashcmp(hash, partial_pack_hash) != 0)
 				die("Unexpected checksum for %s "
 				    "(disk corruption?)", pack_name);
 			/*
@@ -267,23 +267,24 @@ void fixup_pack_header_footer(int pack_fd,
 			 * pack, which also means making partial_pack_offset
 			 * big enough not to matter anymore.
 			 */
-			git_SHA1_Init(&old_sha1_ctx);
+			the_hash_algo->init_fn(&old_hash_ctx);
 			partial_pack_offset = ~partial_pack_offset;
 			partial_pack_offset -= MSB(partial_pack_offset, 1);
 		}
 	}
 	free(buf);
 
-	if (partial_pack_sha1)
-		git_SHA1_Final(partial_pack_sha1, &old_sha1_ctx);
-	git_SHA1_Final(new_pack_sha1, &new_sha1_ctx);
-	write_or_die(pack_fd, new_pack_sha1, 20);
+	if (partial_pack_hash)
+		the_hash_algo->final_fn(partial_pack_hash, &old_hash_ctx);
+	the_hash_algo->final_fn(new_pack_hash, &new_hash_ctx);
+	write_or_die(pack_fd, new_pack_hash, the_hash_algo->rawsz);
 	fsync_or_die(pack_fd, pack_name);
 }
 
 char *index_pack_lockfile(int ip_out)
 {
-	char packname[46];
+	char packname[GIT_MAX_HEXSZ + 6];
+	int len = the_hash_algo->hexsz + 6;
 
 	/*
 	 * The first thing we expect from index-pack's output
@@ -292,9 +293,9 @@ char *index_pack_lockfile(int ip_out)
 	 * case, we need it to remove the corresponding .keep file
 	 * later on.  If we don't get that then tough luck with it.
 	 */
-	if (read_in_full(ip_out, packname, 46) == 46 && packname[45] == '\n') {
+	if (read_in_full(ip_out, packname, len) == len && packname[len-1] == '\n') {
 		const char *name;
-		packname[45] = 0;
+		packname[len-1] = 0;
 		if (skip_prefix(packname, "keep\t", &name))
 			return xstrfmt("%s/pack/pack-%s.keep",
 				       get_object_directory(), name);

Re: [PATCH 08/12] pack-write: switch various SHA-1 values to abstract forms

[Patryk Obara]

On 28/01/2018 16:57, brian m. carlson wrote:
>   		if (partial_pack_offset == 0) {
> -			unsigned char sha1[20];
> -			git_SHA1_Final(sha1, &old_sha1_ctx);
> -			if (hashcmp(sha1, partial_pack_sha1) != 0)
> +			unsigned char hash[GIT_MAX_RAWSZ];
> +			the_hash_algo->final_fn(hash, &old_hash_ctx);
> +			if (hashcmp(hash, partial_pack_hash) != 0)

Maybe "hash" should be struct object_id here?

>   char *index_pack_lockfile(int ip_out)
>   {
> -	char packname[46];
> +	char packname[GIT_MAX_HEXSZ + 6];
> +	int len = the_hash_algo->hexsz + 6;

Just me nitpicking, but "len" can be const :)

-- 
| ← Ceci n'est pas une pipe
Patryk Obara

Re: [PATCH 08/12] pack-write: switch various SHA-1 values to abstract forms

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

On Sun, Jan 28, 2018 at 09:30:36PM +0100, Patryk Obara wrote:
> On 28/01/2018 16:57, brian m. carlson wrote:
> >   		if (partial_pack_offset == 0) {
> > -			unsigned char sha1[20];
> > -			git_SHA1_Final(sha1, &old_sha1_ctx);
> > -			if (hashcmp(sha1, partial_pack_sha1) != 0)
> > +			unsigned char hash[GIT_MAX_RAWSZ];
> > +			the_hash_algo->final_fn(hash, &old_hash_ctx);
> > +			if (hashcmp(hash, partial_pack_hash) != 0)
> 
> Maybe "hash" should be struct object_id here?

In this case, I opted not to do that because it's specifically not an
object ID.  It's a checksum for the pack, which isn't a normal Git
object, so I tried to preserve that distinction.

> >   char *index_pack_lockfile(int ip_out)
> >   {
> > -	char packname[46];
> > +	char packname[GIT_MAX_HEXSZ + 6];
> > +	int len = the_hash_algo->hexsz + 6;
> 
> Just me nitpicking, but "len" can be const :)

I wanted it to be const, too, but I recall getting feedback discouraging
it.  I"m happy to make the change; after all, it can only help the
compiler and any future readers.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]

[PATCH 09/12] read-cache: abstract away uses of SHA-1

[brian m. carlson]

Convert various uses of direct calls to SHA-1 and 20- and 40-based
constants to use the_hash_algo instead.  Don't yet convert the on-disk
data structures, which will be handled in a future commit.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 read-cache.c | 54 +++++++++++++++++++++++++++---------------------------
 1 file changed, 27 insertions(+), 27 deletions(-)

diff --git a/read-cache.c b/read-cache.c
index 2eb81a66b9..4f7aac23af 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -1545,8 +1545,8 @@ int verify_ce_order;
 
 static int verify_hdr(struct cache_header *hdr, unsigned long size)
 {
-	git_SHA_CTX c;
-	unsigned char sha1[20];
+	git_hash_ctx c;
+	unsigned char hash[GIT_MAX_RAWSZ];
 	int hdr_version;
 
 	if (hdr->hdr_signature != htonl(CACHE_SIGNATURE))
@@ -1558,10 +1558,10 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)
 	if (!verify_index_checksum)
 		return 0;
 
-	git_SHA1_Init(&c);
-	git_SHA1_Update(&c, hdr, size - 20);
-	git_SHA1_Final(sha1, &c);
-	if (hashcmp(sha1, (unsigned char *)hdr + size - 20))
+	the_hash_algo->init_fn(&c);
+	the_hash_algo->update_fn(&c, hdr, size - the_hash_algo->rawsz);
+	the_hash_algo->final_fn(hash, &c);
+	if (hashcmp(hash, (unsigned char *)hdr + size - the_hash_algo->rawsz))
 		return error("bad index file sha1 signature");
 	return 0;
 }
@@ -1791,7 +1791,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 		die_errno("cannot stat the open index");
 
 	mmap_size = xsize_t(st.st_size);
-	if (mmap_size < sizeof(struct cache_header) + 20)
+	if (mmap_size < sizeof(struct cache_header) + the_hash_algo->rawsz)
 		die("index file smaller than expected");
 
 	mmap = xmmap(NULL, mmap_size, PROT_READ, MAP_PRIVATE, fd, 0);
@@ -1803,7 +1803,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 	if (verify_hdr(hdr, mmap_size) < 0)
 		goto unmap;
 
-	hashcpy(istate->sha1, (const unsigned char *)hdr + mmap_size - 20);
+	hashcpy(istate->sha1, (const unsigned char *)hdr + mmap_size - the_hash_algo->rawsz);
 	istate->version = ntohl(hdr->hdr_version);
 	istate->cache_nr = ntohl(hdr->hdr_entries);
 	istate->cache_alloc = alloc_nr(istate->cache_nr);
@@ -1831,7 +1831,7 @@ int do_read_index(struct index_state *istate, const char *path, int must_exist)
 	istate->timestamp.sec = st.st_mtime;
 	istate->timestamp.nsec = ST_MTIME_NSEC(st);
 
-	while (src_offset <= mmap_size - 20 - 8) {
+	while (src_offset <= mmap_size - the_hash_algo->rawsz - 8) {
 		/* After an array of active_nr index entries,
 		 * there can be arbitrary number of extended
 		 * sections, each of which is prefixed with
@@ -1957,11 +1957,11 @@ int unmerged_index(const struct index_state *istate)
 static unsigned char write_buffer[WRITE_BUFFER_SIZE];
 static unsigned long write_buffer_len;
 
-static int ce_write_flush(git_SHA_CTX *context, int fd)
+static int ce_write_flush(git_hash_ctx *context, int fd)
 {
 	unsigned int buffered = write_buffer_len;
 	if (buffered) {
-		git_SHA1_Update(context, write_buffer, buffered);
+		the_hash_algo->update_fn(context, write_buffer, buffered);
 		if (write_in_full(fd, write_buffer, buffered) < 0)
 			return -1;
 		write_buffer_len = 0;
@@ -1969,7 +1969,7 @@ static int ce_write_flush(git_SHA_CTX *context, int fd)
 	return 0;
 }
 
-static int ce_write(git_SHA_CTX *context, int fd, void *data, unsigned int len)
+static int ce_write(git_hash_ctx *context, int fd, void *data, unsigned int len)
 {
 	while (len) {
 		unsigned int buffered = write_buffer_len;
@@ -1991,7 +1991,7 @@ static int ce_write(git_SHA_CTX *context, int fd, void *data, unsigned int len)
 	return 0;
 }
 
-static int write_index_ext_header(git_SHA_CTX *context, int fd,
+static int write_index_ext_header(git_hash_ctx *context, int fd,
 				  unsigned int ext, unsigned int sz)
 {
 	ext = htonl(ext);
@@ -2000,26 +2000,26 @@ static int write_index_ext_header(git_SHA_CTX *context, int fd,
 		(ce_write(context, fd, &sz, 4) < 0)) ? -1 : 0;
 }
 
-static int ce_flush(git_SHA_CTX *context, int fd, unsigned char *sha1)
+static int ce_flush(git_hash_ctx *context, int fd, unsigned char *hash)
 {
 	unsigned int left = write_buffer_len;
 
 	if (left) {
 		write_buffer_len = 0;
-		git_SHA1_Update(context, write_buffer, left);
+		the_hash_algo->update_fn(context, write_buffer, left);
 	}
 
 	/* Flush first if not enough space for SHA1 signature */
-	if (left + 20 > WRITE_BUFFER_SIZE) {
+	if (left + the_hash_algo->rawsz > WRITE_BUFFER_SIZE) {
 		if (write_in_full(fd, write_buffer, left) < 0)
 			return -1;
 		left = 0;
 	}
 
 	/* Append the SHA1 signature at the end */
-	git_SHA1_Final(write_buffer + left, context);
-	hashcpy(sha1, write_buffer + left);
-	left += 20;
+	the_hash_algo->final_fn(write_buffer + left, context);
+	hashcpy(hash, write_buffer + left);
+	left += the_hash_algo->rawsz;
 	return (write_in_full(fd, write_buffer, left) < 0) ? -1 : 0;
 }
 
@@ -2100,7 +2100,7 @@ static void copy_cache_entry_to_ondisk(struct ondisk_cache_entry *ondisk,
 	}
 }
 
-static int ce_write_entry(git_SHA_CTX *c, int fd, struct cache_entry *ce,
+static int ce_write_entry(git_hash_ctx *c, int fd, struct cache_entry *ce,
 			  struct strbuf *previous_name, struct ondisk_cache_entry *ondisk)
 {
 	int size;
@@ -2167,7 +2167,7 @@ static int verify_index_from(const struct index_state *istate, const char *path)
 	int fd;
 	ssize_t n;
 	struct stat st;
-	unsigned char sha1[20];
+	unsigned char hash[GIT_MAX_RAWSZ];
 
 	if (!istate->initialized)
 		return 0;
@@ -2179,14 +2179,14 @@ static int verify_index_from(const struct index_state *istate, const char *path)
 	if (fstat(fd, &st))
 		goto out;
 
-	if (st.st_size < sizeof(struct cache_header) + 20)
+	if (st.st_size < sizeof(struct cache_header) + the_hash_algo->rawsz)
 		goto out;
 
-	n = pread_in_full(fd, sha1, 20, st.st_size - 20);
-	if (n != 20)
+	n = pread_in_full(fd, hash, the_hash_algo->rawsz, st.st_size - the_hash_algo->rawsz);
+	if (n != the_hash_algo->rawsz)
 		goto out;
 
-	if (hashcmp(istate->sha1, sha1))
+	if (hashcmp(istate->sha1, hash))
 		goto out;
 
 	close(fd);
@@ -2235,7 +2235,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile,
 			  int strip_extensions)
 {
 	int newfd = tempfile->fd;
-	git_SHA_CTX c;
+	git_hash_ctx c;
 	struct cache_header hdr;
 	int i, err = 0, removed, extended, hdr_version;
 	struct cache_entry **cache = istate->cache;
@@ -2273,7 +2273,7 @@ static int do_write_index(struct index_state *istate, struct tempfile *tempfile,
 	hdr.hdr_version = htonl(hdr_version);
 	hdr.hdr_entries = htonl(entries - removed);
 
-	git_SHA1_Init(&c);
+	the_hash_algo->init_fn(&c);
 	if (ce_write(&c, newfd, &hdr, sizeof(hdr)) < 0)
 		return -1;
 

Re: [PATCH 09/12] read-cache: abstract away uses of SHA-1

[Eric Sunshine]

On Sun, Jan 28, 2018 at 10:57 AM, brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> Convert various uses of direct calls to SHA-1 and 20- and 40-based
> constants to use the_hash_algo instead.  Don't yet convert the on-disk
> data structures, which will be handled in a future commit.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/read-cache.c b/read-cache.c
> @@ -2000,26 +2000,26 @@ static int write_index_ext_header(git_SHA_CTX *context, int fd,
>         /* Flush first if not enough space for SHA1 signature */

Did you want to update the comment to remove the SHA1 reference also?
(Or was the omission intentional per the commit message?)

> -       if (left + 20 > WRITE_BUFFER_SIZE) {
> +       if (left + the_hash_algo->rawsz > WRITE_BUFFER_SIZE) {
>                 if (write_in_full(fd, write_buffer, left) < 0)
>                         return -1;
>                 left = 0;
>         }
>
>         /* Append the SHA1 signature at the end */

Ditto.

> -       git_SHA1_Final(write_buffer + left, context);
> -       hashcpy(sha1, write_buffer + left);
> -       left += 20;
> +       the_hash_algo->final_fn(write_buffer + left, context);
> +       hashcpy(hash, write_buffer + left);
> +       left += the_hash_algo->rawsz;

Re: [PATCH 09/12] read-cache: abstract away uses of SHA-1

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1320 bytes --]

On Sun, Jan 28, 2018 at 02:50:18PM -0500, Eric Sunshine wrote:
> On Sun, Jan 28, 2018 at 10:57 AM, brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
> > Convert various uses of direct calls to SHA-1 and 20- and 40-based
> > constants to use the_hash_algo instead.  Don't yet convert the on-disk
> > data structures, which will be handled in a future commit.
> >
> > Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> > ---
> > diff --git a/read-cache.c b/read-cache.c
> > @@ -2000,26 +2000,26 @@ static int write_index_ext_header(git_SHA_CTX *context, int fd,
> >         /* Flush first if not enough space for SHA1 signature */
> 
> Did you want to update the comment to remove the SHA1 reference also?
> (Or was the omission intentional per the commit message?)

These weren't intentional, so I'll fix them in a reroll.

The only reason I didn't touch the ondisk data structures is because
they're directly mmap'd.  So if I change them to use struct object_id,
as soon as we expand the structure size, the ondisk structures become
unusable.  Some sort of union hack or multiple subroutines will probably
be required.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]

[PATCH 10/12] csum-file: rename sha1file to hashfile

[brian m. carlson]

Rename struct sha1file to struct hashfile, along with all of its related
functions.

The transformation in this commit was made by global search-and-replace.
---
 builtin/index-pack.c   | 20 +++++++++----------
 builtin/pack-objects.c | 52 +++++++++++++++++++++++++-------------------------
 bulk-checkin.c         | 16 ++++++++--------
 csum-file.c            | 36 +++++++++++++++++-----------------
 csum-file.h            | 34 ++++++++++++++++-----------------
 fast-import.c          | 28 +++++++++++++--------------
 pack-bitmap-write.c    | 30 ++++++++++++++---------------
 pack-write.c           | 32 +++++++++++++++----------------
 pack.h                 |  4 ++--
 9 files changed, 126 insertions(+), 126 deletions(-)

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 40c000aca8..0bb520c731 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1241,7 +1241,7 @@ static void resolve_deltas(void)
  * - append objects to convert thin pack to full pack if required
  * - write the final pack hash
  */
-static void fix_unresolved_deltas(struct sha1file *f);
+static void fix_unresolved_deltas(struct hashfile *f);
 static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned char *pack_hash)
 {
 	if (nr_ref_deltas + nr_ofs_deltas == nr_resolved_deltas) {
@@ -1252,7 +1252,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 	}
 
 	if (fix_thin_pack) {
-		struct sha1file *f;
+		struct hashfile *f;
 		unsigned char read_hash[GIT_MAX_RAWSZ], tail_hash[GIT_MAX_RAWSZ];
 		struct strbuf msg = STRBUF_INIT;
 		int nr_unresolved = nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas;
@@ -1262,7 +1262,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 		REALLOC_ARRAY(objects, nr_objects + nr_unresolved + 1);
 		memset(objects + nr_objects + 1, 0,
 		       nr_unresolved * sizeof(*objects));
-		f = sha1fd(output_fd, curr_pack);
+		f = hashfd(output_fd, curr_pack);
 		fix_unresolved_deltas(f);
 		strbuf_addf(&msg, Q_("completed with %d local object",
 				     "completed with %d local objects",
@@ -1270,7 +1270,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 			    nr_objects - nr_objects_initial);
 		stop_progress_msg(&progress, msg.buf);
 		strbuf_release(&msg);
-		sha1close(f, tail_hash, 0);
+		hashclose(f, tail_hash, 0);
 		hashcpy(read_hash, pack_hash);
 		fixup_pack_header_footer(output_fd, pack_hash,
 					 curr_pack, nr_objects,
@@ -1286,7 +1286,7 @@ static void conclude_pack(int fix_thin_pack, const char *curr_pack, unsigned cha
 		    nr_ofs_deltas + nr_ref_deltas - nr_resolved_deltas);
 }
 
-static int write_compressed(struct sha1file *f, void *in, unsigned int size)
+static int write_compressed(struct hashfile *f, void *in, unsigned int size)
 {
 	git_zstream stream;
 	int status;
@@ -1300,7 +1300,7 @@ static int write_compressed(struct sha1file *f, void *in, unsigned int size)
 		stream.next_out = outbuf;
 		stream.avail_out = sizeof(outbuf);
 		status = git_deflate(&stream, Z_FINISH);
-		sha1write(f, outbuf, sizeof(outbuf) - stream.avail_out);
+		hashwrite(f, outbuf, sizeof(outbuf) - stream.avail_out);
 	} while (status == Z_OK);
 
 	if (status != Z_STREAM_END)
@@ -1310,7 +1310,7 @@ static int write_compressed(struct sha1file *f, void *in, unsigned int size)
 	return size;
 }
 
-static struct object_entry *append_obj_to_pack(struct sha1file *f,
+static struct object_entry *append_obj_to_pack(struct hashfile *f,
 			       const unsigned char *sha1, void *buf,
 			       unsigned long size, enum object_type type)
 {
@@ -1327,7 +1327,7 @@ static struct object_entry *append_obj_to_pack(struct sha1file *f,
 	}
 	header[n++] = c;
 	crc32_begin(f);
-	sha1write(f, header, n);
+	hashwrite(f, header, n);
 	obj[0].size = size;
 	obj[0].hdr_size = n;
 	obj[0].type = type;
@@ -1335,7 +1335,7 @@ static struct object_entry *append_obj_to_pack(struct sha1file *f,
 	obj[1].idx.offset = obj[0].idx.offset + n;
 	obj[1].idx.offset += write_compressed(f, buf, size);
 	obj[0].idx.crc32 = crc32_end(f);
-	sha1flush(f);
+	hashflush(f);
 	hashcpy(obj->idx.oid.hash, sha1);
 	return obj;
 }
@@ -1347,7 +1347,7 @@ static int delta_pos_compare(const void *_a, const void *_b)
 	return a->obj_no - b->obj_no;
 }
 
-static void fix_unresolved_deltas(struct sha1file *f)
+static void fix_unresolved_deltas(struct hashfile *f)
 {
 	struct ref_delta_entry **sorted_by_pos;
 	int i;
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 6b9cfc289d..d91cbd460e 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -161,7 +161,7 @@ static unsigned long do_compress(void **pptr, unsigned long size)
 	return stream.total_out;
 }
 
-static unsigned long write_large_blob_data(struct git_istream *st, struct sha1file *f,
+static unsigned long write_large_blob_data(struct git_istream *st, struct hashfile *f,
 					   const struct object_id *oid)
 {
 	git_zstream stream;
@@ -185,7 +185,7 @@ static unsigned long write_large_blob_data(struct git_istream *st, struct sha1fi
 			stream.next_out = obuf;
 			stream.avail_out = sizeof(obuf);
 			zret = git_deflate(&stream, readlen ? 0 : Z_FINISH);
-			sha1write(f, obuf, stream.next_out - obuf);
+			hashwrite(f, obuf, stream.next_out - obuf);
 			olen += stream.next_out - obuf;
 		}
 		if (stream.avail_in)
@@ -230,7 +230,7 @@ static int check_pack_inflate(struct packed_git *p,
 		stream.total_in == len) ? 0 : -1;
 }
 
-static void copy_pack_data(struct sha1file *f,
+static void copy_pack_data(struct hashfile *f,
 		struct packed_git *p,
 		struct pack_window **w_curs,
 		off_t offset,
@@ -243,14 +243,14 @@ static void copy_pack_data(struct sha1file *f,
 		in = use_pack(p, w_curs, offset, &avail);
 		if (avail > len)
 			avail = (unsigned long)len;
-		sha1write(f, in, avail);
+		hashwrite(f, in, avail);
 		offset += avail;
 		len -= avail;
 	}
 }
 
 /* Return 0 if we will bust the pack-size limit */
-static unsigned long write_no_reuse_object(struct sha1file *f, struct object_entry *entry,
+static unsigned long write_no_reuse_object(struct hashfile *f, struct object_entry *entry,
 					   unsigned long limit, int usable_delta)
 {
 	unsigned long size, datalen;
@@ -323,8 +323,8 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 			free(buf);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
-		sha1write(f, dheader + pos, sizeof(dheader) - pos);
+		hashwrite(f, header, hdrlen);
+		hashwrite(f, dheader + pos, sizeof(dheader) - pos);
 		hdrlen += sizeof(dheader) - pos;
 	} else if (type == OBJ_REF_DELTA) {
 		/*
@@ -337,8 +337,8 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 			free(buf);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
-		sha1write(f, entry->delta->idx.oid.hash, 20);
+		hashwrite(f, header, hdrlen);
+		hashwrite(f, entry->delta->idx.oid.hash, 20);
 		hdrlen += 20;
 	} else {
 		if (limit && hdrlen + datalen + 20 >= limit) {
@@ -347,13 +347,13 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 			free(buf);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
+		hashwrite(f, header, hdrlen);
 	}
 	if (st) {
 		datalen = write_large_blob_data(st, f, &entry->idx.oid);
 		close_istream(st);
 	} else {
-		sha1write(f, buf, datalen);
+		hashwrite(f, buf, datalen);
 		free(buf);
 	}
 
@@ -361,7 +361,7 @@ static unsigned long write_no_reuse_object(struct sha1file *f, struct object_ent
 }
 
 /* Return 0 if we will bust the pack-size limit */
-static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
+static off_t write_reuse_object(struct hashfile *f, struct object_entry *entry,
 				unsigned long limit, int usable_delta)
 {
 	struct packed_git *p = entry->in_pack;
@@ -412,8 +412,8 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
 			unuse_pack(&w_curs);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
-		sha1write(f, dheader + pos, sizeof(dheader) - pos);
+		hashwrite(f, header, hdrlen);
+		hashwrite(f, dheader + pos, sizeof(dheader) - pos);
 		hdrlen += sizeof(dheader) - pos;
 		reused_delta++;
 	} else if (type == OBJ_REF_DELTA) {
@@ -421,8 +421,8 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
 			unuse_pack(&w_curs);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
-		sha1write(f, entry->delta->idx.oid.hash, 20);
+		hashwrite(f, header, hdrlen);
+		hashwrite(f, entry->delta->idx.oid.hash, 20);
 		hdrlen += 20;
 		reused_delta++;
 	} else {
@@ -430,7 +430,7 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
 			unuse_pack(&w_curs);
 			return 0;
 		}
-		sha1write(f, header, hdrlen);
+		hashwrite(f, header, hdrlen);
 	}
 	copy_pack_data(f, p, &w_curs, offset, datalen);
 	unuse_pack(&w_curs);
@@ -439,7 +439,7 @@ static off_t write_reuse_object(struct sha1file *f, struct object_entry *entry,
 }
 
 /* Return 0 if we will bust the pack-size limit */
-static off_t write_object(struct sha1file *f,
+static off_t write_object(struct hashfile *f,
 			  struct object_entry *entry,
 			  off_t write_offset)
 {
@@ -512,7 +512,7 @@ enum write_one_status {
 	WRITE_ONE_RECURSIVE = 2 /* already scheduled to be written */
 };
 
-static enum write_one_status write_one(struct sha1file *f,
+static enum write_one_status write_one(struct hashfile *f,
 				       struct object_entry *e,
 				       off_t *offset)
 {
@@ -731,7 +731,7 @@ static struct object_entry **compute_write_order(void)
 	return wo;
 }
 
-static off_t write_reused_pack(struct sha1file *f)
+static off_t write_reused_pack(struct hashfile *f)
 {
 	unsigned char buffer[8192];
 	off_t to_write, total;
@@ -762,7 +762,7 @@ static off_t write_reused_pack(struct sha1file *f)
 		if (read_pack > to_write)
 			read_pack = to_write;
 
-		sha1write(f, buffer, read_pack);
+		hashwrite(f, buffer, read_pack);
 		to_write -= read_pack;
 
 		/*
@@ -791,7 +791,7 @@ static const char no_split_warning[] = N_(
 static void write_pack_file(void)
 {
 	uint32_t i = 0, j;
-	struct sha1file *f;
+	struct hashfile *f;
 	off_t offset;
 	uint32_t nr_remaining = nr_result;
 	time_t last_mtime = 0;
@@ -807,7 +807,7 @@ static void write_pack_file(void)
 		char *pack_tmp_name = NULL;
 
 		if (pack_to_stdout)
-			f = sha1fd_throughput(1, "<stdout>", progress_state);
+			f = hashfd_throughput(1, "<stdout>", progress_state);
 		else
 			f = create_tmp_packfile(&pack_tmp_name);
 
@@ -834,11 +834,11 @@ static void write_pack_file(void)
 		 * If so, rewrite it like in fast-import
 		 */
 		if (pack_to_stdout) {
-			sha1close(f, oid.hash, CSUM_CLOSE);
+			hashclose(f, oid.hash, CSUM_CLOSE);
 		} else if (nr_written == nr_remaining) {
-			sha1close(f, oid.hash, CSUM_FSYNC);
+			hashclose(f, oid.hash, CSUM_FSYNC);
 		} else {
-			int fd = sha1close(f, oid.hash, 0);
+			int fd = hashclose(f, oid.hash, 0);
 			fixup_pack_header_footer(fd, oid.hash, pack_tmp_name,
 						 nr_written, oid.hash, offset);
 			close(fd);
diff --git a/bulk-checkin.c b/bulk-checkin.c
index 3310fd210a..5f79ed6ea3 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -12,7 +12,7 @@ static struct bulk_checkin_state {
 	unsigned plugged:1;
 
 	char *pack_tmp_name;
-	struct sha1file *f;
+	struct hashfile *f;
 	off_t offset;
 	struct pack_idx_option pack_idx_opts;
 
@@ -35,9 +35,9 @@ static void finish_bulk_checkin(struct bulk_checkin_state *state)
 		unlink(state->pack_tmp_name);
 		goto clear_exit;
 	} else if (state->nr_written == 1) {
-		sha1close(state->f, oid.hash, CSUM_FSYNC);
+		hashclose(state->f, oid.hash, CSUM_FSYNC);
 	} else {
-		int fd = sha1close(state->f, oid.hash, 0);
+		int fd = hashclose(state->f, oid.hash, 0);
 		fixup_pack_header_footer(fd, oid.hash, state->pack_tmp_name,
 					 state->nr_written, oid.hash,
 					 state->offset);
@@ -149,7 +149,7 @@ static int stream_to_pack(struct bulk_checkin_state *state,
 					return -1;
 				}
 
-				sha1write(state->f, obuf, written);
+				hashwrite(state->f, obuf, written);
 				state->offset += written;
 			}
 			s.next_out = obuf;
@@ -195,7 +195,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 	git_SHA_CTX ctx;
 	unsigned char obuf[16384];
 	unsigned header_len;
-	struct sha1file_checkpoint checkpoint;
+	struct hashfile_checkpoint checkpoint;
 	struct pack_idx_entry *idx = NULL;
 
 	seekback = lseek(fd, 0, SEEK_CUR);
@@ -216,7 +216,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 	while (1) {
 		prepare_to_stream(state, flags);
 		if (idx) {
-			sha1file_checkpoint(state->f, &checkpoint);
+			hashfile_checkpoint(state->f, &checkpoint);
 			idx->offset = state->offset;
 			crc32_begin(state->f);
 		}
@@ -230,7 +230,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 		 */
 		if (!idx)
 			die("BUG: should not happen");
-		sha1file_truncate(state->f, &checkpoint);
+		hashfile_truncate(state->f, &checkpoint);
 		state->offset = checkpoint.offset;
 		finish_bulk_checkin(state);
 		if (lseek(fd, seekback, SEEK_SET) == (off_t) -1)
@@ -242,7 +242,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 
 	idx->crc32 = crc32_end(state->f);
 	if (already_written(state, result_sha1)) {
-		sha1file_truncate(state->f, &checkpoint);
+		hashfile_truncate(state->f, &checkpoint);
 		state->offset = checkpoint.offset;
 		free(idx);
 	} else {
diff --git a/csum-file.c b/csum-file.c
index 2adae04073..e4ad6337dc 100644
--- a/csum-file.c
+++ b/csum-file.c
@@ -11,7 +11,7 @@
 #include "progress.h"
 #include "csum-file.h"
 
-static void flush(struct sha1file *f, const void *buf, unsigned int count)
+static void flush(struct hashfile *f, const void *buf, unsigned int count)
 {
 	if (0 <= f->check_fd && count)  {
 		unsigned char check_buffer[8192];
@@ -42,7 +42,7 @@ static void flush(struct sha1file *f, const void *buf, unsigned int count)
 	}
 }
 
-void sha1flush(struct sha1file *f)
+void hashflush(struct hashfile *f)
 {
 	unsigned offset = f->offset;
 
@@ -53,11 +53,11 @@ void sha1flush(struct sha1file *f)
 	}
 }
 
-int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags)
+int hashclose(struct hashfile *f, unsigned char *result, unsigned int flags)
 {
 	int fd;
 
-	sha1flush(f);
+	hashflush(f);
 	git_SHA1_Final(f->buffer, &f->ctx);
 	if (result)
 		hashcpy(result, f->buffer);
@@ -86,7 +86,7 @@ int sha1close(struct sha1file *f, unsigned char *result, unsigned int flags)
 	return fd;
 }
 
-void sha1write(struct sha1file *f, const void *buf, unsigned int count)
+void hashwrite(struct hashfile *f, const void *buf, unsigned int count)
 {
 	while (count) {
 		unsigned offset = f->offset;
@@ -118,15 +118,15 @@ void sha1write(struct sha1file *f, const void *buf, unsigned int count)
 	}
 }
 
-struct sha1file *sha1fd(int fd, const char *name)
+struct hashfile *hashfd(int fd, const char *name)
 {
-	return sha1fd_throughput(fd, name, NULL);
+	return hashfd_throughput(fd, name, NULL);
 }
 
-struct sha1file *sha1fd_check(const char *name)
+struct hashfile *hashfd_check(const char *name)
 {
 	int sink, check;
-	struct sha1file *f;
+	struct hashfile *f;
 
 	sink = open("/dev/null", O_WRONLY);
 	if (sink < 0)
@@ -134,14 +134,14 @@ struct sha1file *sha1fd_check(const char *name)
 	check = open(name, O_RDONLY);
 	if (check < 0)
 		die_errno("unable to open '%s'", name);
-	f = sha1fd(sink, name);
+	f = hashfd(sink, name);
 	f->check_fd = check;
 	return f;
 }
 
-struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp)
+struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp)
 {
-	struct sha1file *f = xmalloc(sizeof(*f));
+	struct hashfile *f = xmalloc(sizeof(*f));
 	f->fd = fd;
 	f->check_fd = -1;
 	f->offset = 0;
@@ -153,14 +153,14 @@ struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp
 	return f;
 }
 
-void sha1file_checkpoint(struct sha1file *f, struct sha1file_checkpoint *checkpoint)
+void hashfile_checkpoint(struct hashfile *f, struct hashfile_checkpoint *checkpoint)
 {
-	sha1flush(f);
+	hashflush(f);
 	checkpoint->offset = f->total;
 	checkpoint->ctx = f->ctx;
 }
 
-int sha1file_truncate(struct sha1file *f, struct sha1file_checkpoint *checkpoint)
+int hashfile_truncate(struct hashfile *f, struct hashfile_checkpoint *checkpoint)
 {
 	off_t offset = checkpoint->offset;
 
@@ -169,17 +169,17 @@ int sha1file_truncate(struct sha1file *f, struct sha1file_checkpoint *checkpoint
 		return -1;
 	f->total = offset;
 	f->ctx = checkpoint->ctx;
-	f->offset = 0; /* sha1flush() was called in checkpoint */
+	f->offset = 0; /* hashflush() was called in checkpoint */
 	return 0;
 }
 
-void crc32_begin(struct sha1file *f)
+void crc32_begin(struct hashfile *f)
 {
 	f->crc32 = crc32(0, NULL, 0);
 	f->do_crc = 1;
 }
 
-uint32_t crc32_end(struct sha1file *f)
+uint32_t crc32_end(struct hashfile *f)
 {
 	f->do_crc = 0;
 	return f->crc32;
diff --git a/csum-file.h b/csum-file.h
index 7530927d77..ceb3e5712d 100644
--- a/csum-file.h
+++ b/csum-file.h
@@ -4,7 +4,7 @@
 struct progress;
 
 /* A SHA1-protected file */
-struct sha1file {
+struct hashfile {
 	int fd;
 	int check_fd;
 	unsigned int offset;
@@ -18,36 +18,36 @@ struct sha1file {
 };
 
 /* Checkpoint */
-struct sha1file_checkpoint {
+struct hashfile_checkpoint {
 	off_t offset;
 	git_SHA_CTX ctx;
 };
 
-extern void sha1file_checkpoint(struct sha1file *, struct sha1file_checkpoint *);
-extern int sha1file_truncate(struct sha1file *, struct sha1file_checkpoint *);
+extern void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *);
+extern int hashfile_truncate(struct hashfile *, struct hashfile_checkpoint *);
 
-/* sha1close flags */
+/* hashclose flags */
 #define CSUM_CLOSE	1
 #define CSUM_FSYNC	2
 
-extern struct sha1file *sha1fd(int fd, const char *name);
-extern struct sha1file *sha1fd_check(const char *name);
-extern struct sha1file *sha1fd_throughput(int fd, const char *name, struct progress *tp);
-extern int sha1close(struct sha1file *, unsigned char *, unsigned int);
-extern void sha1write(struct sha1file *, const void *, unsigned int);
-extern void sha1flush(struct sha1file *f);
-extern void crc32_begin(struct sha1file *);
-extern uint32_t crc32_end(struct sha1file *);
+extern struct hashfile *hashfd(int fd, const char *name);
+extern struct hashfile *hashfd_check(const char *name);
+extern struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp);
+extern int hashclose(struct hashfile *, unsigned char *, unsigned int);
+extern void hashwrite(struct hashfile *, const void *, unsigned int);
+extern void hashflush(struct hashfile *f);
+extern void crc32_begin(struct hashfile *);
+extern uint32_t crc32_end(struct hashfile *);
 
-static inline void sha1write_u8(struct sha1file *f, uint8_t data)
+static inline void hashwrite_u8(struct hashfile *f, uint8_t data)
 {
-	sha1write(f, &data, sizeof(data));
+	hashwrite(f, &data, sizeof(data));
 }
 
-static inline void sha1write_be32(struct sha1file *f, uint32_t data)
+static inline void hashwrite_be32(struct hashfile *f, uint32_t data)
 {
 	data = htonl(data);
-	sha1write(f, &data, sizeof(data));
+	hashwrite(f, &data, sizeof(data));
 }
 
 #endif
diff --git a/fast-import.c b/fast-import.c
index 1b8ab8ea29..26e071b3d5 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -316,7 +316,7 @@ static struct atom_str **atom_table;
 /* The .pack file being generated */
 static struct pack_idx_option pack_idx_opts;
 static unsigned int pack_id;
-static struct sha1file *pack_file;
+static struct hashfile *pack_file;
 static struct packed_git *pack_data;
 static struct packed_git **all_packs;
 static off_t pack_size;
@@ -905,12 +905,12 @@ static void start_packfile(void)
 
 	p->pack_fd = pack_fd;
 	p->do_not_close = 1;
-	pack_file = sha1fd(pack_fd, p->pack_name);
+	pack_file = hashfd(pack_fd, p->pack_name);
 
 	hdr.hdr_signature = htonl(PACK_SIGNATURE);
 	hdr.hdr_version = htonl(2);
 	hdr.hdr_entries = 0;
-	sha1write(pack_file, &hdr, sizeof(hdr));
+	hashwrite(pack_file, &hdr, sizeof(hdr));
 
 	pack_data = p;
 	pack_size = sizeof(hdr);
@@ -1016,7 +1016,7 @@ static void end_packfile(void)
 		struct tag *t;
 
 		close_pack_windows(pack_data);
-		sha1close(pack_file, cur_pack_oid.hash, 0);
+		hashclose(pack_file, cur_pack_oid.hash, 0);
 		fixup_pack_header_footer(pack_data->pack_fd, pack_data->sha1,
 				    pack_data->pack_name, object_count,
 				    cur_pack_oid.hash, pack_size);
@@ -1180,23 +1180,23 @@ static int store_object(
 
 		hdrlen = encode_in_pack_object_header(hdr, sizeof(hdr),
 						      OBJ_OFS_DELTA, deltalen);
-		sha1write(pack_file, hdr, hdrlen);
+		hashwrite(pack_file, hdr, hdrlen);
 		pack_size += hdrlen;
 
 		hdr[pos] = ofs & 127;
 		while (ofs >>= 7)
 			hdr[--pos] = 128 | (--ofs & 127);
-		sha1write(pack_file, hdr + pos, sizeof(hdr) - pos);
+		hashwrite(pack_file, hdr + pos, sizeof(hdr) - pos);
 		pack_size += sizeof(hdr) - pos;
 	} else {
 		e->depth = 0;
 		hdrlen = encode_in_pack_object_header(hdr, sizeof(hdr),
 						      type, dat->len);
-		sha1write(pack_file, hdr, hdrlen);
+		hashwrite(pack_file, hdr, hdrlen);
 		pack_size += hdrlen;
 	}
 
-	sha1write(pack_file, out, s.total_out);
+	hashwrite(pack_file, out, s.total_out);
 	pack_size += s.total_out;
 
 	e->idx.crc32 = crc32_end(pack_file);
@@ -1215,9 +1215,9 @@ static int store_object(
 	return 0;
 }
 
-static void truncate_pack(struct sha1file_checkpoint *checkpoint)
+static void truncate_pack(struct hashfile_checkpoint *checkpoint)
 {
-	if (sha1file_truncate(pack_file, checkpoint))
+	if (hashfile_truncate(pack_file, checkpoint))
 		die_errno("cannot truncate pack to skip duplicate");
 	pack_size = checkpoint->offset;
 }
@@ -1233,7 +1233,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 	off_t offset;
 	git_hash_ctx c;
 	git_zstream s;
-	struct sha1file_checkpoint checkpoint;
+	struct hashfile_checkpoint checkpoint;
 	int status = Z_OK;
 
 	/* Determine if we should auto-checkpoint. */
@@ -1241,7 +1241,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 		|| (pack_size + 60 + len) < pack_size)
 		cycle_packfile();
 
-	sha1file_checkpoint(pack_file, &checkpoint);
+	hashfile_checkpoint(pack_file, &checkpoint);
 	offset = checkpoint.offset;
 
 	hdrlen = xsnprintf((char *)out_buf, out_sz, "blob %" PRIuMAX, len) + 1;
@@ -1275,7 +1275,7 @@ static void stream_blob(uintmax_t len, struct object_id *oidout, uintmax_t mark)
 
 		if (!s.avail_out || status == Z_STREAM_END) {
 			size_t n = s.next_out - out_buf;
-			sha1write(pack_file, out_buf, n);
+			hashwrite(pack_file, out_buf, n);
 			pack_size += n;
 			s.next_out = out_buf;
 			s.avail_out = out_sz;
@@ -1360,7 +1360,7 @@ static void *gfi_unpack_entry(
 		 * the newly written data.
 		 */
 		close_pack_windows(p);
-		sha1flush(pack_file);
+		hashflush(pack_file);
 
 		/* We have to offer rawsz bytes additional on the end of
 		 * the packfile as the core unpacker code assumes the
diff --git a/pack-bitmap-write.c b/pack-bitmap-write.c
index a8df5ce2ab..e01f992884 100644
--- a/pack-bitmap-write.c
+++ b/pack-bitmap-write.c
@@ -440,19 +440,19 @@ void bitmap_writer_select_commits(struct commit **indexed_commits,
 }
 
 
-static int sha1write_ewah_helper(void *f, const void *buf, size_t len)
+static int hashwrite_ewah_helper(void *f, const void *buf, size_t len)
 {
-	/* sha1write will die on error */
-	sha1write(f, buf, len);
+	/* hashwrite will die on error */
+	hashwrite(f, buf, len);
 	return len;
 }
 
 /**
  * Write the bitmap index to disk
  */
-static inline void dump_bitmap(struct sha1file *f, struct ewah_bitmap *bitmap)
+static inline void dump_bitmap(struct hashfile *f, struct ewah_bitmap *bitmap)
 {
-	if (ewah_serialize_to(bitmap, sha1write_ewah_helper, f) < 0)
+	if (ewah_serialize_to(bitmap, hashwrite_ewah_helper, f) < 0)
 		die("Failed to write bitmap index");
 }
 
@@ -462,7 +462,7 @@ static const unsigned char *sha1_access(size_t pos, void *table)
 	return index[pos]->oid.hash;
 }
 
-static void write_selected_commits_v1(struct sha1file *f,
+static void write_selected_commits_v1(struct hashfile *f,
 				      struct pack_idx_entry **index,
 				      uint32_t index_nr)
 {
@@ -477,15 +477,15 @@ static void write_selected_commits_v1(struct sha1file *f,
 		if (commit_pos < 0)
 			die("BUG: trying to write commit not in index");
 
-		sha1write_be32(f, commit_pos);
-		sha1write_u8(f, stored->xor_offset);
-		sha1write_u8(f, stored->flags);
+		hashwrite_be32(f, commit_pos);
+		hashwrite_u8(f, stored->xor_offset);
+		hashwrite_u8(f, stored->flags);
 
 		dump_bitmap(f, stored->write_as);
 	}
 }
 
-static void write_hash_cache(struct sha1file *f,
+static void write_hash_cache(struct hashfile *f,
 			     struct pack_idx_entry **index,
 			     uint32_t index_nr)
 {
@@ -494,7 +494,7 @@ static void write_hash_cache(struct sha1file *f,
 	for (i = 0; i < index_nr; ++i) {
 		struct object_entry *entry = (struct object_entry *)index[i];
 		uint32_t hash_value = htonl(entry->hash);
-		sha1write(f, &hash_value, sizeof(hash_value));
+		hashwrite(f, &hash_value, sizeof(hash_value));
 	}
 }
 
@@ -511,13 +511,13 @@ void bitmap_writer_finish(struct pack_idx_entry **index,
 	static uint16_t default_version = 1;
 	static uint16_t flags = BITMAP_OPT_FULL_DAG;
 	struct strbuf tmp_file = STRBUF_INIT;
-	struct sha1file *f;
+	struct hashfile *f;
 
 	struct bitmap_disk_header header;
 
 	int fd = odb_mkstemp(&tmp_file, "pack/tmp_bitmap_XXXXXX");
 
-	f = sha1fd(fd, tmp_file.buf);
+	f = hashfd(fd, tmp_file.buf);
 
 	memcpy(header.magic, BITMAP_IDX_SIGNATURE, sizeof(BITMAP_IDX_SIGNATURE));
 	header.version = htons(default_version);
@@ -525,7 +525,7 @@ void bitmap_writer_finish(struct pack_idx_entry **index,
 	header.entry_count = htonl(writer.selected_nr);
 	hashcpy(header.checksum, writer.pack_checksum);
 
-	sha1write(f, &header, sizeof(header));
+	hashwrite(f, &header, sizeof(header));
 	dump_bitmap(f, writer.commits);
 	dump_bitmap(f, writer.trees);
 	dump_bitmap(f, writer.blobs);
@@ -535,7 +535,7 @@ void bitmap_writer_finish(struct pack_idx_entry **index,
 	if (options & BITMAP_OPT_HASH_CACHE)
 		write_hash_cache(f, index, index_nr);
 
-	sha1close(f, NULL, CSUM_FSYNC);
+	hashclose(f, NULL, CSUM_FSYNC);
 
 	if (adjust_shared_perm(tmp_file.buf))
 		die_errno("unable to make temporary bitmap file readable");
diff --git a/pack-write.c b/pack-write.c
index fe33f7464c..7c256b5e80 100644
--- a/pack-write.c
+++ b/pack-write.c
@@ -46,7 +46,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 			   int nr_objects, const struct pack_idx_option *opts,
 			   const unsigned char *sha1)
 {
-	struct sha1file *f;
+	struct hashfile *f;
 	struct pack_idx_entry **sorted_by_sha, **list, **last;
 	off_t last_obj_offset = 0;
 	uint32_t array[256];
@@ -68,7 +68,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 
 	if (opts->flags & WRITE_IDX_VERIFY) {
 		assert(index_name);
-		f = sha1fd_check(index_name);
+		f = hashfd_check(index_name);
 	} else {
 		if (!index_name) {
 			struct strbuf tmp_file = STRBUF_INIT;
@@ -80,7 +80,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 			if (fd < 0)
 				die_errno("unable to create '%s'", index_name);
 		}
-		f = sha1fd(fd, index_name);
+		f = hashfd(fd, index_name);
 	}
 
 	/* if last object's offset is >= 2^31 we should use index V2 */
@@ -91,7 +91,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		struct pack_idx_header hdr;
 		hdr.idx_signature = htonl(PACK_IDX_SIGNATURE);
 		hdr.idx_version = htonl(index_version);
-		sha1write(f, &hdr, sizeof(hdr));
+		hashwrite(f, &hdr, sizeof(hdr));
 	}
 
 	/*
@@ -110,7 +110,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		array[i] = htonl(next - sorted_by_sha);
 		list = next;
 	}
-	sha1write(f, array, 256 * 4);
+	hashwrite(f, array, 256 * 4);
 
 	/*
 	 * Write the actual SHA1 entries..
@@ -120,9 +120,9 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		struct pack_idx_entry *obj = *list++;
 		if (index_version < 2) {
 			uint32_t offset = htonl(obj->offset);
-			sha1write(f, &offset, 4);
+			hashwrite(f, &offset, 4);
 		}
-		sha1write(f, obj->oid.hash, the_hash_algo->rawsz);
+		hashwrite(f, obj->oid.hash, the_hash_algo->rawsz);
 		if ((opts->flags & WRITE_IDX_STRICT) &&
 		    (i && !oidcmp(&list[-2]->oid, &obj->oid)))
 			die("The same object %s appears twice in the pack",
@@ -137,7 +137,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 		for (i = 0; i < nr_objects; i++) {
 			struct pack_idx_entry *obj = *list++;
 			uint32_t crc32_val = htonl(obj->crc32);
-			sha1write(f, &crc32_val, 4);
+			hashwrite(f, &crc32_val, 4);
 		}
 
 		/* write the 32-bit offset table */
@@ -150,7 +150,7 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 				  ? (0x80000000 | nr_large_offset++)
 				  : obj->offset);
 			offset = htonl(offset);
-			sha1write(f, &offset, 4);
+			hashwrite(f, &offset, 4);
 		}
 
 		/* write the large offset table */
@@ -164,25 +164,25 @@ const char *write_idx_file(const char *index_name, struct pack_idx_entry **objec
 				continue;
 			split[0] = htonl(offset >> 32);
 			split[1] = htonl(offset & 0xffffffff);
-			sha1write(f, split, 8);
+			hashwrite(f, split, 8);
 			nr_large_offset--;
 		}
 	}
 
-	sha1write(f, sha1, the_hash_algo->rawsz);
-	sha1close(f, NULL, ((opts->flags & WRITE_IDX_VERIFY)
+	hashwrite(f, sha1, the_hash_algo->rawsz);
+	hashclose(f, NULL, ((opts->flags & WRITE_IDX_VERIFY)
 			    ? CSUM_CLOSE : CSUM_FSYNC));
 	return index_name;
 }
 
-off_t write_pack_header(struct sha1file *f, uint32_t nr_entries)
+off_t write_pack_header(struct hashfile *f, uint32_t nr_entries)
 {
 	struct pack_header hdr;
 
 	hdr.hdr_signature = htonl(PACK_SIGNATURE);
 	hdr.hdr_version = htonl(PACK_VERSION);
 	hdr.hdr_entries = htonl(nr_entries);
-	sha1write(f, &hdr, sizeof(hdr));
+	hashwrite(f, &hdr, sizeof(hdr));
 	return sizeof(hdr);
 }
 
@@ -333,14 +333,14 @@ int encode_in_pack_object_header(unsigned char *hdr, int hdr_len,
 	return n;
 }
 
-struct sha1file *create_tmp_packfile(char **pack_tmp_name)
+struct hashfile *create_tmp_packfile(char **pack_tmp_name)
 {
 	struct strbuf tmpname = STRBUF_INIT;
 	int fd;
 
 	fd = odb_mkstemp(&tmpname, "pack/tmp_pack_XXXXXX");
 	*pack_tmp_name = strbuf_detach(&tmpname, NULL);
-	return sha1fd(fd, *pack_tmp_name);
+	return hashfd(fd, *pack_tmp_name);
 }
 
 void finish_tmp_packfile(struct strbuf *name_buffer,
diff --git a/pack.h b/pack.h
index 8294341af1..34a9d458b4 100644
--- a/pack.h
+++ b/pack.h
@@ -81,7 +81,7 @@ extern const char *write_idx_file(const char *index_name, struct pack_idx_entry
 extern int check_pack_crc(struct packed_git *p, struct pack_window **w_curs, off_t offset, off_t len, unsigned int nr);
 extern int verify_pack_index(struct packed_git *);
 extern int verify_pack(struct packed_git *, verify_fn fn, struct progress *, uint32_t);
-extern off_t write_pack_header(struct sha1file *f, uint32_t);
+extern off_t write_pack_header(struct hashfile *f, uint32_t);
 extern void fixup_pack_header_footer(int, unsigned char *, const char *, uint32_t, unsigned char *, off_t);
 extern char *index_pack_lockfile(int fd);
 
@@ -98,7 +98,7 @@ extern int encode_in_pack_object_header(unsigned char *hdr, int hdr_len,
 #define PH_ERROR_PROTOCOL	(-3)
 extern int read_pack_header(int fd, struct pack_header *);
 
-extern struct sha1file *create_tmp_packfile(char **pack_tmp_name);
+extern struct hashfile *create_tmp_packfile(char **pack_tmp_name);
 extern void finish_tmp_packfile(struct strbuf *name_buffer, const char *pack_tmp_name, struct pack_idx_entry **written_list, uint32_t nr_written, struct pack_idx_option *pack_idx_opts, unsigned char sha1[]);
 
 #endif

[PATCH 11/12] csum-file: abstract uses of SHA-1

[brian m. carlson]

Convert several direct uses of SHA-1 to use the_hash_algo instead.
Convert one use of the constant 20 as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 csum-file.c | 10 +++++-----
 csum-file.h |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/csum-file.c b/csum-file.c
index e4ad6337dc..5eda7fb6af 100644
--- a/csum-file.c
+++ b/csum-file.c
@@ -47,7 +47,7 @@ void hashflush(struct hashfile *f)
 	unsigned offset = f->offset;
 
 	if (offset) {
-		git_SHA1_Update(&f->ctx, f->buffer, offset);
+		the_hash_algo->update_fn(&f->ctx, f->buffer, offset);
 		flush(f, f->buffer, offset);
 		f->offset = 0;
 	}
@@ -58,12 +58,12 @@ int hashclose(struct hashfile *f, unsigned char *result, unsigned int flags)
 	int fd;
 
 	hashflush(f);
-	git_SHA1_Final(f->buffer, &f->ctx);
+	the_hash_algo->final_fn(f->buffer, &f->ctx);
 	if (result)
 		hashcpy(result, f->buffer);
 	if (flags & (CSUM_CLOSE | CSUM_FSYNC)) {
 		/* write checksum and close fd */
-		flush(f, f->buffer, 20);
+		flush(f, f->buffer, the_hash_algo->rawsz);
 		if (flags & CSUM_FSYNC)
 			fsync_or_die(f->fd, f->name);
 		if (close(f->fd))
@@ -110,7 +110,7 @@ void hashwrite(struct hashfile *f, const void *buf, unsigned int count)
 		buf = (char *) buf + nr;
 		left -= nr;
 		if (!left) {
-			git_SHA1_Update(&f->ctx, data, offset);
+			the_hash_algo->update_fn(&f->ctx, data, offset);
 			flush(f, data, offset);
 			offset = 0;
 		}
@@ -149,7 +149,7 @@ struct hashfile *hashfd_throughput(int fd, const char *name, struct progress *tp
 	f->tp = tp;
 	f->name = name;
 	f->do_crc = 0;
-	git_SHA1_Init(&f->ctx);
+	the_hash_algo->init_fn(&f->ctx);
 	return f;
 }
 
diff --git a/csum-file.h b/csum-file.h
index ceb3e5712d..992e5c0141 100644
--- a/csum-file.h
+++ b/csum-file.h
@@ -8,7 +8,7 @@ struct hashfile {
 	int fd;
 	int check_fd;
 	unsigned int offset;
-	git_SHA_CTX ctx;
+	git_hash_ctx ctx;
 	off_t total;
 	struct progress *tp;
 	const char *name;
@@ -20,7 +20,7 @@ struct hashfile {
 /* Checkpoint */
 struct hashfile_checkpoint {
 	off_t offset;
-	git_SHA_CTX ctx;
+	git_hash_ctx ctx;
 };
 
 extern void hashfile_checkpoint(struct hashfile *, struct hashfile_checkpoint *);

[PATCH 12/12] bulk-checkin: abstract SHA-1 usage

[brian m. carlson]

Convert uses of the direct SHA-1 functions to use the_hash_algo instead.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 bulk-checkin.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/bulk-checkin.c b/bulk-checkin.c
index 5f79ed6ea3..8bcd1c8665 100644
--- a/bulk-checkin.c
+++ b/bulk-checkin.c
@@ -93,7 +93,7 @@ static int already_written(struct bulk_checkin_state *state, unsigned char sha1[
  * with a new pack.
  */
 static int stream_to_pack(struct bulk_checkin_state *state,
-			  git_SHA_CTX *ctx, off_t *already_hashed_to,
+			  git_hash_ctx *ctx, off_t *already_hashed_to,
 			  int fd, size_t size, enum object_type type,
 			  const char *path, unsigned flags)
 {
@@ -127,7 +127,7 @@ static int stream_to_pack(struct bulk_checkin_state *state,
 				if (rsize < hsize)
 					hsize = rsize;
 				if (hsize)
-					git_SHA1_Update(ctx, ibuf, hsize);
+					the_hash_algo->update_fn(ctx, ibuf, hsize);
 				*already_hashed_to = offset;
 			}
 			s.next_in = ibuf;
@@ -192,7 +192,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 			   unsigned flags)
 {
 	off_t seekback, already_hashed_to;
-	git_SHA_CTX ctx;
+	git_hash_ctx ctx;
 	unsigned char obuf[16384];
 	unsigned header_len;
 	struct hashfile_checkpoint checkpoint;
@@ -204,8 +204,8 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 
 	header_len = xsnprintf((char *)obuf, sizeof(obuf), "%s %" PRIuMAX,
 			       typename(type), (uintmax_t)size) + 1;
-	git_SHA1_Init(&ctx);
-	git_SHA1_Update(&ctx, obuf, header_len);
+	the_hash_algo->init_fn(&ctx);
+	the_hash_algo->update_fn(&ctx, obuf, header_len);
 
 	/* Note: idx is non-NULL when we are writing */
 	if ((flags & HASH_WRITE_OBJECT) != 0)
@@ -236,7 +236,7 @@ static int deflate_to_pack(struct bulk_checkin_state *state,
 		if (lseek(fd, seekback, SEEK_SET) == (off_t) -1)
 			return error("cannot seek back");
 	}
-	git_SHA1_Final(result_sha1, &ctx);
+	the_hash_algo->final_fn(result_sha1, &ctx);
 	if (!idx)
 		return 0;
 

Re: [PATCH 00/12] object_id part 11 (the_hash_algo)

[Patryk Obara]

I looked at your branch object-id-part-12, and it conflicts with my next
batch of object_id conversions in quite many places (mostly through
formatting). Therefore I'll hold my horses and postpone my conversion
patches at least until part 12 will be sent.

-- 
| ← Ceci n'est pas une pipe
Patryk Obara

Re: [PATCH 00/12] object_id part 11 (the_hash_algo)

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 651 bytes --]

On Sun, Jan 28, 2018 at 09:48:28PM +0100, Patryk Obara wrote:
> I looked at your branch object-id-part-12, and it conflicts with my next
> batch of object_id conversions in quite many places (mostly through
> formatting). Therefore I'll hold my horses and postpone my conversion
> patches at least until part 12 will be sent.

Okay.  I've just rebased it on top of your object_id series, so I'll
reroll this series and then once both series get picked up, rebase and
submit it on top.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
https://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 867 bytes --]