git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "brian m. carlson" <sandals@crustytoothpaste.net>
To: git@vger.kernel.org
Cc: "Derrick Stolee" <stolee@gmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
	"Duy Nguyen" <pclouds@gmail.com>,
	"SZEDER Gábor" <szeder.dev@gmail.com>,
	"Jakub Narebski" <jnareb@gmail.com>,
	"Christian Couder" <christian.couder@gmail.com>
Subject: [PATCH v5 00/12] Base SHA-256 implementation
Date: Sun,  4 Nov 2018 23:44:46 +0000	[thread overview]
Message-ID: <20181104234458.139223-1-sandals@crustytoothpaste.net> (raw)
In-Reply-To: <20181025024005.154208-1-sandals@crustytoothpaste.net>

This series provides a functional SHA-256 implementation and wires it
up, along with some housekeeping patches to make it suitable for
testing.

Changes from v4:
* Downcase hex constants for consistency.
* Remove needless parentheses in return statement.
* Remove braces for single statement loops.
* Switch to +=.
* Add references to rationale for SHA-256.
* Remove inclusion of "git-compat-util.h" in header.

Changes from v3:
* Switch to using inline functions instead of macros in many cases.
* Undefine remaining macros at the top.

Changes from v2:
* Improve commit messages to include timing and performance information.
* Improve commit messages to be less ambiguous and more friendly to a
  wider variety of English speakers.
* Prefer functions taking struct git_hash_algo in hex.c.
* Port pieces of the block-sha1 implementation over to the block-sha256
  implementation for better compatibility.
* Drop patch 13 in favor of further discussion about the best way
  forward for versioning commit graph.
* Rename the test so as to have a different number from other tests.
* Rebase on master.

Changes from v1:
* Add a hash_to_hex function mirroring sha1_to_hex, but for
  the_hash_algo.
* Strip commit message explanation about why we chose SHA-256.
* Rebase on master
* Strip leading whitespace from commit message.
* Improve commit-graph patch to cover new code added since v1.
* Be more honest about the scope of work involved in porting the SHA-256
  implementation out of libtomcrypt.
* Revert change to limit hashcmp to 20 bytes.

brian m. carlson (12):
  sha1-file: rename algorithm to "sha1"
  sha1-file: provide functions to look up hash algorithms
  hex: introduce functions to print arbitrary hashes
  cache: make hashcmp and hasheq work with larger hashes
  t: add basic tests for our SHA-1 implementation
  t: make the sha1 test-tool helper generic
  sha1-file: add a constant for hash block size
  t/helper: add a test helper to compute hash speed
  commit-graph: convert to using the_hash_algo
  Add a base implementation of SHA-256 support
  sha256: add an SHA-256 implementation using libgcrypt
  hash: add an SHA-256 implementation using OpenSSL

 Makefile                              |  22 +++
 cache.h                               |  51 ++++---
 commit-graph.c                        |  33 ++---
 hash.h                                |  41 +++++-
 hex.c                                 |  32 +++--
 sha1-file.c                           |  70 ++++++++-
 sha256/block/sha256.c                 | 196 ++++++++++++++++++++++++++
 sha256/block/sha256.h                 |  26 ++++
 sha256/gcrypt.h                       |  30 ++++
 t/helper/test-hash-speed.c            |  61 ++++++++
 t/helper/{test-sha1.c => test-hash.c} |  19 +--
 t/helper/test-sha1.c                  |  52 +------
 t/helper/test-sha256.c                |   7 +
 t/helper/test-tool.c                  |   2 +
 t/helper/test-tool.h                  |   4 +
 t/t0015-hash.sh                       |  54 +++++++
 16 files changed, 596 insertions(+), 104 deletions(-)
 create mode 100644 sha256/block/sha256.c
 create mode 100644 sha256/block/sha256.h
 create mode 100644 sha256/gcrypt.h
 create mode 100644 t/helper/test-hash-speed.c
 copy t/helper/{test-sha1.c => test-hash.c} (65%)
 create mode 100644 t/helper/test-sha256.c
 create mode 100755 t/t0015-hash.sh

Range-diff against v4:
 1:  a004a4c982 <  -:  ---------- :hash-impl
 2:  cf9f7f5620 =  1:  cf9f7f5620 sha1-file: rename algorithm to "sha1"
 3:  0144deaebe =  2:  0144deaebe sha1-file: provide functions to look up hash algorithms
 4:  b74858fb03 =  3:  b74858fb03 hex: introduce functions to print arbitrary hashes
 5:  e9703017a4 =  4:  e9703017a4 cache: make hashcmp and hasheq work with larger hashes
 6:  ab85a834fd =  5:  ab85a834fd t: add basic tests for our SHA-1 implementation
 7:  962f6d8903 =  6:  962f6d8903 t: make the sha1 test-tool helper generic
 8:  53addf4d58 =  7:  53addf4d58 sha1-file: add a constant for hash block size
 9:  9ace10faa2 =  8:  9ace10faa2 t/helper: add a test helper to compute hash speed
10:  9adc56d01e =  9:  9adc56d01e commit-graph: convert to using the_hash_algo
11:  f48cb1ad27 ! 10:  90544c504c Add a base implementation of SHA-256 support
    @@ -4,7 +4,9 @@
     
         SHA-1 is weak and we need to transition to a new hash function.  For
         some time, we have referred to this new function as NewHash.  Recently,
    -    we decided to pick SHA-256 as NewHash.
    +    we decided to pick SHA-256 as NewHash.  The reasons behind the choice of
    +    SHA-256 are outlined in the thread starting at [1] and in the commit
    +    history for the hash function transition document.
     
         Add a basic implementation of SHA-256 based off libtomcrypt, which is in
         the public domain.  Optimize it and restructure it to meet our coding
    @@ -20,6 +22,8 @@
         SHA-256 in Git.  Additional patches are needed to prepare the code to
         handle a larger hash algorithm and further test fixes are needed.
     
    +    [1] https://public-inbox.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/
    +
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      diff --git a/Makefile b/Makefile
    @@ -216,14 +220,14 @@
     +{
     +	ctx->offset = 0;
     +	ctx->size = 0;
    -+	ctx->state[0] = 0x6A09E667UL;
    -+	ctx->state[1] = 0xBB67AE85UL;
    -+	ctx->state[2] = 0x3C6EF372UL;
    -+	ctx->state[3] = 0xA54FF53AUL;
    -+	ctx->state[4] = 0x510E527FUL;
    -+	ctx->state[5] = 0x9B05688CUL;
    -+	ctx->state[6] = 0x1F83D9ABUL;
    -+	ctx->state[7] = 0x5BE0CD19UL;
    ++	ctx->state[0] = 0x6a09e667ul;
    ++	ctx->state[1] = 0xbb67ae85ul;
    ++	ctx->state[2] = 0x3c6ef372ul;
    ++	ctx->state[3] = 0xa54ff53aul;
    ++	ctx->state[4] = 0x510e527ful;
    ++	ctx->state[5] = 0x9b05688cul;
    ++	ctx->state[6] = 0x1f83d9abul;
    ++	ctx->state[7] = 0x5be0cd19ul;
     +}
     +
     +static inline uint32_t ror(uint32_t x, unsigned n)
    @@ -233,12 +237,12 @@
     +
     +static inline uint32_t ch(uint32_t x, uint32_t y, uint32_t z)
     +{
    -+	return (z ^ (x & (y ^ z)));
    ++	return z ^ (x & (y ^ z));
     +}
     +
     +static inline uint32_t maj(uint32_t x, uint32_t y, uint32_t z)
     +{
    -+	return (((x | y) & z) | (x & y));
    ++	return ((x | y) & z) | (x & y);
     +}
     +
     +static inline uint32_t sigma0(uint32_t x)
    @@ -268,19 +272,16 @@
     +	int i;
     +
     +	/* copy state into S */
    -+	for (i = 0; i < 8; i++) {
    ++	for (i = 0; i < 8; i++)
     +		S[i] = ctx->state[i];
    -+	}
     +
     +	/* copy the state into 512-bits into W[0..15] */
    -+	for (i = 0; i < 16; i++, buf += sizeof(uint32_t)) {
    ++	for (i = 0; i < 16; i++, buf += sizeof(uint32_t))
     +		W[i] = get_be32(buf);
    -+	}
     +
     +	/* fill W[16..63] */
    -+	for (i = 16; i < 64; i++) {
    ++	for (i = 16; i < 64; i++)
     +		W[i] = gamma1(W[i - 2]) + W[i - 7] + gamma0(W[i - 15]) + W[i - 16];
    -+	}
     +
     +#define RND(a,b,c,d,e,f,g,h,i,ki)                    \
     +	t0 = h + sigma1(e) + ch(e, f, g) + ki + W[i];   \
    @@ -353,10 +354,8 @@
     +	RND(S[2],S[3],S[4],S[5],S[6],S[7],S[0],S[1],62,0xbef9a3f7);
     +	RND(S[1],S[2],S[3],S[4],S[5],S[6],S[7],S[0],63,0xc67178f2);
     +
    -+
    -+	for (i = 0; i < 8; i++) {
    -+		ctx->state[i] = ctx->state[i] + S[i];
    -+	}
    ++	for (i = 0; i < 8; i++)
    ++		ctx->state[i] += S[i];
     +}
     +
     +void blk_SHA256_Update(blk_SHA256_CTX *ctx, const void *data, size_t len)
12:  fe8f2ba01c = 11:  467c86e878 sha256: add an SHA-256 implementation using libgcrypt
13:  38142d8fc6 = 12:  73e4bc17d0 hash: add an SHA-256 implementation using OpenSSL

  parent reply	other threads:[~2018-11-04 23:45 UTC|newest]

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-25  2:39 [PATCH v4 00/12] Base SHA-256 implementation brian m. carlson
2018-10-25  2:39 ` [PATCH v4 01/12] sha1-file: rename algorithm to "sha1" brian m. carlson
2018-10-25  2:39 ` [PATCH v4 02/12] sha1-file: provide functions to look up hash algorithms brian m. carlson
2018-10-25  2:39 ` [PATCH v4 03/12] hex: introduce functions to print arbitrary hashes brian m. carlson
2018-10-25  2:39 ` [PATCH v4 04/12] cache: make hashcmp and hasheq work with larger hashes brian m. carlson
2018-10-25  2:39 ` [PATCH v4 05/12] t: add basic tests for our SHA-1 implementation brian m. carlson
2018-10-25  2:39 ` [PATCH v4 06/12] t: make the sha1 test-tool helper generic brian m. carlson
2018-10-25  2:40 ` [PATCH v4 07/12] sha1-file: add a constant for hash block size brian m. carlson
2018-10-25  2:40 ` [PATCH v4 08/12] t/helper: add a test helper to compute hash speed brian m. carlson
2018-10-25  2:40 ` [PATCH v4 09/12] commit-graph: convert to using the_hash_algo brian m. carlson
2018-10-25  2:40 ` [PATCH v4 10/12] Add a base implementation of SHA-256 support brian m. carlson
2018-10-25  3:02   ` Carlo Arenas
2018-10-28 15:52     ` brian m. carlson
2018-10-29  0:39       ` Junio C Hamano
2018-10-31 22:55         ` brian m. carlson
2018-11-01  5:29           ` Junio C Hamano
2018-10-27  9:03   ` Christian Couder
2018-10-25  2:40 ` [PATCH v4 11/12] sha256: add an SHA-256 implementation using libgcrypt brian m. carlson
2018-10-25  2:40 ` [PATCH v4 12/12] hash: add an SHA-256 implementation using OpenSSL brian m. carlson
2018-11-04 23:44 ` brian m. carlson [this message]
2018-11-04 23:44   ` [PATCH v5 01/12] sha1-file: rename algorithm to "sha1" brian m. carlson
2018-11-05  7:21     ` Ævar Arnfjörð Bjarmason
2018-11-04 23:44   ` [PATCH v5 02/12] sha1-file: provide functions to look up hash algorithms brian m. carlson
2018-11-13 18:42     ` Derrick Stolee
2018-11-13 18:45       ` Duy Nguyen
2018-11-14  1:01         ` brian m. carlson
2018-11-14  0:11       ` Ramsay Jones
2018-11-14  0:42         ` Ramsay Jones
2018-11-14  0:51           ` Jeff King
2018-11-14  2:11         ` brian m. carlson
2018-11-14  3:53           ` Ramsay Jones
2018-11-04 23:44   ` [PATCH v5 03/12] hex: introduce functions to print arbitrary hashes brian m. carlson
2018-11-04 23:44   ` [PATCH v5 04/12] cache: make hashcmp and hasheq work with larger hashes brian m. carlson
2018-11-04 23:44   ` [PATCH v5 05/12] t: add basic tests for our SHA-1 implementation brian m. carlson
2018-11-04 23:44   ` [PATCH v5 06/12] t: make the sha1 test-tool helper generic brian m. carlson
2018-11-04 23:44   ` [PATCH v5 07/12] sha1-file: add a constant for hash block size brian m. carlson
2018-11-04 23:44   ` [PATCH v5 08/12] t/helper: add a test helper to compute hash speed brian m. carlson
2018-11-04 23:44   ` [PATCH v5 09/12] commit-graph: convert to using the_hash_algo brian m. carlson
2018-11-04 23:44   ` [PATCH v5 10/12] Add a base implementation of SHA-256 support brian m. carlson
2018-11-05 11:39     ` Ævar Arnfjörð Bjarmason
2018-11-07  1:30       ` brian m. carlson
2018-11-10 15:52         ` Ævar Arnfjörð Bjarmason
2018-11-04 23:44   ` [PATCH v5 11/12] sha256: add an SHA-256 implementation using libgcrypt brian m. carlson
2018-11-04 23:44   ` [PATCH v5 12/12] hash: add an SHA-256 implementation using OpenSSL brian m. carlson
2018-11-05  2:45   ` [PATCH v5 00/12] Base SHA-256 implementation Junio C Hamano
2018-11-14  4:09   ` [PATCH v6 " brian m. carlson
2018-11-14  4:09     ` [PATCH v6 01/12] sha1-file: rename algorithm to "sha1" brian m. carlson
2018-11-14  4:09     ` [PATCH v6 02/12] sha1-file: provide functions to look up hash algorithms brian m. carlson
2018-11-14  4:09     ` [PATCH v6 03/12] hex: introduce functions to print arbitrary hashes brian m. carlson
2018-11-14  4:09     ` [PATCH v6 04/12] cache: make hashcmp and hasheq work with larger hashes brian m. carlson
2018-11-14  4:09     ` [PATCH v6 05/12] t: add basic tests for our SHA-1 implementation brian m. carlson
2018-11-14  4:09     ` [PATCH v6 06/12] t: make the sha1 test-tool helper generic brian m. carlson
2018-11-14  4:09     ` [PATCH v6 07/12] sha1-file: add a constant for hash block size brian m. carlson
2018-11-14  4:09     ` [PATCH v6 08/12] t/helper: add a test helper to compute hash speed brian m. carlson
2018-11-14  4:09     ` [PATCH v6 09/12] commit-graph: convert to using the_hash_algo brian m. carlson
2018-11-14  4:09     ` [PATCH v6 10/12] Add a base implementation of SHA-256 support brian m. carlson
2018-11-14  4:09     ` [PATCH v6 11/12] sha256: add an SHA-256 implementation using libgcrypt brian m. carlson
2018-11-14  4:09     ` [PATCH v6 12/12] hash: add an SHA-256 implementation using OpenSSL brian m. carlson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181104234458.139223-1-sandals@crustytoothpaste.net \
    --to=sandals@crustytoothpaste.net \
    --cc=avarab@gmail.com \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=jnareb@gmail.com \
    --cc=pclouds@gmail.com \
    --cc=stolee@gmail.com \
    --cc=szeder.dev@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).