git@vger.kernel.org mailing list mirror (one of many)
 help / Atom feed
From: Jonathan Nieder <jrnieder@gmail.com>
To: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Edward Thomson <ethomson@edwardthomson.com>,
	"brian m . carlson" <sandals@crustytoothpaste.net>,
	Johannes Schindelin <Johannes.Schindelin@gmx.de>,
	demerphq <demerphq@gmail.com>, Adam Langley <agl@google.com>,
	keccak@noekeon.org
Subject: Re: [PATCH v2 2/2] doc hash-function-transition: pick SHA-256 as NewHash
Date: Fri, 3 Aug 2018 00:20:14 -0700
Message-ID: <20180803072014.GA256410@aiede.svl.corp.google.com> (raw)
In-Reply-To: <20180726134111.17623-1-avarab@gmail.com>

Hi again,

Sorry for the slow review.  I finally got a chance to look this over
again.

My main nits are about the commit message: I think it still focuses
too much on the process instead of the usual "knowing what I know now,
here's the clearest explanation for why we need this patch" approach.
I can send a patch illustrating what I mean tomorrow morning.

Ævar Arnfjörð Bjarmason wrote:

> From a security perspective, it seems that SHA-256, BLAKE2, SHA3-256,
> K12, and so on are all believed to have similar security properties.
> All are good options from a security point of view.
>
> SHA-256 has a number of advantages:
>
> * It has been around for a while, is widely used, and is supported by
>   just about every single crypto library (OpenSSL, mbedTLS, CryptoNG,
>   SecureTransport, etc).
>
> * When you compare against SHA1DC, most vectorized SHA-256
>   implementations are indeed faster, even without acceleration.
>
> * If we're doing signatures with OpenPGP (or even, I suppose, CMS),
>   we're going to be using SHA-2, so it doesn't make sense to have our
>   security depend on two separate algorithms when either one of them
>   alone could break the security when we could just depend on one.
>
> So SHA-256 it is.

The above is what I wrote, so of course I'd like it. ;-)

>                    See the "Hash algorithm analysis" thread as of
> [1]. Linus has come around to this choice and suggested Junio make the
> final pick, and he's endorsed SHA-256 [3].

The above paragraph has the same problem as before of (1) not being
self-contained and (2) focusing on the process that led to this patch
instead of the benefit of the patch itself.  I think we should omit it.

> This follow-up change changes occurrences of "NewHash" to
> "SHA-256" (or "sha256", depending on the context). The "Selection of a
> New Hash" section has also been changed to note that historically we
> used the the "NewHash" name while we didn't know what the new hash
> function would be.

nit: Commit messages are usually in the imperative tense.  This is in
the past tense, I think because it is a continuation of that
discussion about process.

For this part, I think we can let the patch speak for itself.

> This leaves no use of "NewHash" anywhere in git.git except in the
> aforementioned section (and as a variable name in t/t9700/test.pl, but
> that use from 2008 has nothing to do with this transition plan).

This part is helpful --- good.

> 1. https://public-inbox.org/git/20180720215220.GB18502@genre.crustytoothpaste.net/
> 2. https://public-inbox.org/git/CA+55aFwSe9BF8e0hLk9pp3FVD5LaVY5GRdsV3fbNtgzekJadyA@mail.gmail.com/
> 3. https://public-inbox.org/git/xmqqzhygwd5o.fsf@gitster-ct.c.googlers.com/

Footnotes to the historical part --- I'd recommend removing these.

> Helped-by: Jonathan Nieder <jrnieder@gmail.com>
> Helped-by: Junio C Hamano <gitster@pobox.com>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

Here I'd want to put a pile of acks, e.g.:

 Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
 Acked-by: brian m. carlson <sandals@crustytoothpaste.net>
 Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
 Acked-by: Dan Shumow <danshu@microsoft.com>
 Acked-by: Junio C Hamano <gitster@pobox.com>

[...]
> --- a/Documentation/technical/hash-function-transition.txt
> +++ b/Documentation/technical/hash-function-transition.txt
> @@ -59,14 +59,11 @@ that are believed to be cryptographically secure.
>  
>  Goals
>  -----
> -Where NewHash is a strong 256-bit hash function to replace SHA-1 (see
> -"Selection of a New Hash", below):
> -
> -1. The transition to NewHash can be done one local repository at a time.
> +1. The transition to SHA-256 can be done one local repository at a time.

Yay!

[...]
>  	[extensions]
> -		objectFormat = newhash
> +		objectFormat = sha256
>  		compatObjectFormat = sha1

Yes, makes sense.

[...]
> @@ -155,36 +152,36 @@ repository extensions.
>  Object names
>  ~~~~~~~~~~~~
>  Objects can be named by their 40 hexadecimal digit sha1-name or 64
> -hexadecimal digit newhash-name, plus names derived from those (see
> +hexadecimal digit sha256-name, plus names derived from those (see
>  gitrevisions(7)).
>  
>  The sha1-name of an object is the SHA-1 of the concatenation of its
>  type, length, a nul byte, and the object's sha1-content. This is the
>  traditional <sha1> used in Git to name objects.
>  
> -The newhash-name of an object is the NewHash of the concatenation of its
> -type, length, a nul byte, and the object's newhash-content.
> +The sha256-name of an object is the SHA-256 of the concatenation of its
> +type, length, a nul byte, and the object's sha256-content.

Sensible.

[...]
>  
>  Object format
>  ~~~~~~~~~~~~~
>  The content as a byte sequence of a tag, commit, or tree object named
> -by sha1 and newhash differ because an object named by newhash-name refers to
> +by sha1 and sha256 differ because an object named by sha256-name refers to

Not about this patch: this should say SHA-1 and SHA-256, I think.
Leaving it as is in this patch as you did is the right thing.

[...]
> @@ -255,10 +252,10 @@ network byte order):
>    up to and not including the table of CRC32 values.
>  - Zero or more NUL bytes.
>  - The trailer consists of the following:
> -  - A copy of the 20-byte NewHash checksum at the end of the
> +  - A copy of the 20-byte SHA-256 checksum at the end of the

Not about this patch: a SHA-256 is 32 bytes.  Leaving that for a
separate patch as you did is the right thing, though.

[...]
> -  - 20-byte NewHash checksum of all of the above.
> +  - 20-byte SHA-256 checksum of all of the above.

Likewise.

[...]
> @@ -351,12 +348,12 @@ the following steps:
>     (This list only contains objects reachable from the "wants". If the
>     pack from the server contained additional extraneous objects, then
>     they will be discarded.)
> -3. convert to newhash: open a new (newhash) packfile. Read the topologically
> +3. convert to sha256: open a new (sha256) packfile. Read the topologically

Not about this patch: this one's more murky, since it's talking about
the object names instead of the hash function.  I guess "sha256"
instead of "SHA-256" for this could be right, but I worry it's going
to take time for me to figure out the exact distinction.  That seems
like a reason to just call it SHA-256 (but in a separate patch).

[...]
> -   sha1-content, convert to newhash-content, and write it to the newhash
> -   pack. Record the new sha1<->newhash mapping entry for use in the idx.
> +   sha1-content, convert to sha256-content, and write it to the sha256
> +   pack. Record the new sha1<->sha256 mapping entry for use in the idx.
>  4. sort: reorder entries in the new pack to match the order of objects
> -   in the pack the server generated and include blobs. Write a newhash idx
> +   in the pack the server generated and include blobs. Write a sha256 idx
>     file

Likewise.

[...]
> @@ -388,16 +385,16 @@ send-pack.
>  
>  Signed Commits
>  ~~~~~~~~~~~~~~
> -We add a new field "gpgsig-newhash" to the commit object format to allow
> +We add a new field "gpgsig-sha256" to the commit object format to allow
>  signing commits without relying on SHA-1. It is similar to the
> -existing "gpgsig" field. Its signed payload is the newhash-content of the
> -commit object with any "gpgsig" and "gpgsig-newhash" fields removed.
> +existing "gpgsig" field. Its signed payload is the sha256-content of the
> +commit object with any "gpgsig" and "gpgsig-sha256" fields removed.

That reminds me --- we need to add support for stripping these out.

[...]
> @@ -601,18 +598,22 @@ The user can also explicitly specify which format to use for a
>  particular revision specifier and for output, overriding the mode. For
>  example:
>  
> -git --output-format=sha1 log abac87a^{sha1}..f787cac^{newhash}
> +git --output-format=sha1 log abac87a^{sha1}..f787cac^{sha256}
>  
> -Selection of a New Hash
> ------------------------
> +Choice of Hash
> +--------------

Yay!

[...]
> -Some hashes under consideration are SHA-256, SHA-512/256, SHA-256x16,
> -K12, and BLAKE2bp-256.
> +We choose SHA-256. See the thread starting at
> +<20180609224913.GC38834@genre.crustytoothpaste.net> for the discussion
> +(https://public-inbox.org/git/20180609224913.GC38834@genre.crustytoothpaste.net/)

Can this reference be moved to a footnote?  It's not part of the
design, but it's a good reference.

Thanks again for getting this documented.

Sincerely,
Jonathan

  reply index

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-06-09 20:56 State of NewHash work, future directions, and discussion brian m. carlson
2018-06-09 21:26 ` Ævar Arnfjörð Bjarmason
2018-06-09 22:49 ` Hash algorithm analysis brian m. carlson
2018-06-11 19:29   ` Jonathan Nieder
2018-06-11 20:20     ` Linus Torvalds
2018-06-11 23:27       ` Ævar Arnfjörð Bjarmason
2018-06-12  0:11         ` David Lang
2018-06-12  0:45         ` Linus Torvalds
2018-06-11 22:35     ` brian m. carlson
2018-06-12 16:21       ` Gilles Van Assche
2018-06-13 23:58         ` brian m. carlson
2018-06-15 10:33           ` Gilles Van Assche
2018-07-20 21:52     ` brian m. carlson
2018-07-21  0:31       ` Jonathan Nieder
2018-07-21 19:52       ` Ævar Arnfjörð Bjarmason
2018-07-21 20:25         ` brian m. carlson
2018-07-21 22:38       ` Johannes Schindelin
2018-07-21 23:09         ` Linus Torvalds
2018-07-21 23:59         ` brian m. carlson
2018-07-22  9:34           ` Eric Deplagne
2018-07-22 14:21             ` brian m. carlson
2018-07-22 14:55               ` Eric Deplagne
2018-07-26 10:05                 ` Johannes Schindelin
2018-07-22 15:23           ` Joan Daemen
2018-07-22 18:54             ` Adam Langley
2018-07-26 10:31             ` Johannes Schindelin
2018-07-23 12:40           ` demerphq
2018-07-23 12:48             ` Sitaram Chamarty
2018-07-23 12:55               ` demerphq
2018-07-23 18:23               ` Linus Torvalds
2018-07-23 17:57             ` Stefan Beller
2018-07-23 18:35             ` Jonathan Nieder
2018-07-24 19:01       ` Edward Thomson
2018-07-24 20:31         ` Linus Torvalds
2018-07-24 20:49           ` Jonathan Nieder
2018-07-24 21:13           ` Junio C Hamano
2018-07-24 22:10             ` brian m. carlson
2018-07-30  9:06               ` Johannes Schindelin
2018-07-30 20:01                 ` Dan Shumow
2018-08-03  2:57                   ` Jonathan Nieder
2018-09-18 15:18                   ` Joan Daemen
2018-09-18 15:32                     ` Jonathan Nieder
2018-09-18 16:50                     ` Linus Torvalds
2018-07-25  8:30             ` [PATCH 0/2] document that NewHash is now SHA-256 Ævar Arnfjörð Bjarmason
2018-07-25  8:30             ` [PATCH 1/2] doc hash-function-transition: note the lack of a changelog Ævar Arnfjörð Bjarmason
2018-07-25  8:30             ` [PATCH 2/2] doc hash-function-transition: pick SHA-256 as NewHash Ævar Arnfjörð Bjarmason
2018-07-25 16:45               ` Junio C Hamano
2018-07-25 17:25                 ` Jonathan Nieder
2018-07-25 21:32                   ` Junio C Hamano
2018-07-26 13:41                     ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2018-08-03  7:20                       ` Jonathan Nieder [this message]
2018-08-03 16:40                         ` Junio C Hamano
2018-08-03 17:01                           ` Linus Torvalds
2018-08-03 16:42                         ` Linus Torvalds
2018-08-03 17:43                         ` Ævar Arnfjörð Bjarmason
2018-08-04  8:52                           ` Jonathan Nieder
2018-08-03 17:45                         ` brian m. carlson
2018-07-25 22:56                 ` [PATCH " brian m. carlson
2018-06-11 21:19   ` Hash algorithm analysis Ævar Arnfjörð Bjarmason
2018-06-21  8:20     ` Johannes Schindelin
2018-06-21 22:39     ` brian m. carlson
2018-06-11 18:09 ` State of NewHash work, future directions, and discussion Duy Nguyen
2018-06-12  1:28   ` brian m. carlson
2018-06-11 19:01 ` Jonathan Nieder
2018-06-12  2:28   ` brian m. carlson
2018-06-12  2:42     ` Jonathan Nieder

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180803072014.GA256410@aiede.svl.corp.google.com \
    --to=jrnieder@gmail.com \
    --cc=Johannes.Schindelin@gmx.de \
    --cc=agl@google.com \
    --cc=avarab@gmail.com \
    --cc=demerphq@gmail.com \
    --cc=ethomson@edwardthomson.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=keccak@noekeon.org \
    --cc=sandals@crustytoothpaste.net \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org mailing list mirror (one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox