git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Johannes Schindelin <johannes.schindelin@gmx.de>
To: git@vger.kernel.org
Cc: Junio C Hamano <gitster@pobox.com>
Subject: [PATCH 7/7] p0013: new test to compare SHA1DC vs OpenSSL
Date: Sat, 25 Mar 2017 00:28:44 +0100 (CET)	[thread overview]
Message-ID: <4c2e3b2abc84a4b4eb7b41b424f76defda550646.1490397869.git.johannes.schindelin@gmx.de> (raw)
In-Reply-To: <cover.1490397869.git.johannes.schindelin@gmx.de>

To demonstrate the need for the core.enableSHA1DC knob, this test
compares the performance of the SHA-1 algorithms with collision
detection vs OpenSSL's (that does not detect attempted collision
attacks).

The payload size of 300MB was actually not concocted from thin air, but
is based on the massive Windows monorepo, whose index file weighs in
with roughly that size (and which is SHA-1'ed upon every single read and
write).

On this developer's machine, this comparison shows a hefty difference:

	0013.1: calculate SHA-1 for 300MB (SHA1DC)    3.03(0.03+0.17)
	0013.2: calculate SHA-1 for 300MB (OpenSSL)   0.58(0.06+0.16)

It is not only that ~6x slower performance is a pretty tall order, the
absolute numbers themselves speak a very clear language: having to wait
one second every time a file is `git add`ed is noticeable, but one can
handwave it away. Having to wait six seconds (3 to read the index, a
fraction of a millisecond to hash the new contents and update the
in-memory index, then 3 seconds to write out the index) is outright
annoying. And unnecessary, too: the content of the index is never
crafted to cause SHA-1 collisions.

Obviously, this test requires that Git was built with the new
DC_AND_OPENSSL_SHA1 make flag; it is skipped otherwise.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/perf/p0013-sha1dc.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 t/perf/p0013-sha1dc.sh

diff --git a/t/perf/p0013-sha1dc.sh b/t/perf/p0013-sha1dc.sh
new file mode 100644
index 00000000000..e08473ac969
--- /dev/null
+++ b/t/perf/p0013-sha1dc.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+
+test_description="Tests performance of SHA1DC vs OpenSSL"
+
+. ./perf-lib.sh
+
+test -n "$DC_AND_OPENSSL_SHA1" || {
+	skip_all='DC_AND_OPENSSL_SHA1 required for this test'
+	test_done
+}
+
+test_perf 'calculate SHA-1 for 300MB (SHA1DC)' '
+	dd if=/dev/zero bs=1M count=300 |
+	test-sha1
+'
+
+test_perf 'calculate SHA-1 for 300MB (OpenSSL)' '
+	dd if=/dev/zero bs=1M count=300 |
+	test-sha1 --disable-sha1dc
+'
+
+test_done
+
-- 
2.12.1.windows.1

  parent reply	other threads:[~2017-03-24 23:28 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-24 23:24 [PATCH 0/7] PREVIEW: Introduce DC_AND_OPENSSL_SHA1 make flag Johannes Schindelin
2017-03-24 23:24 ` [PATCH 1/7] sha1dc: safeguard against outside definitions of BIGENDIAN Johannes Schindelin
2017-03-24 23:24 ` [PATCH 2/7] Makefile: optionally compile with both SHA1DC and SHA1_OPENSSL Johannes Schindelin
2017-03-25 19:51   ` Ævar Arnfjörð Bjarmason
2017-03-30 16:16   ` Junio C Hamano
2017-03-30 16:47     ` Junio C Hamano
2017-04-18 11:28     ` Johannes Schindelin
2017-03-24 23:24 ` [PATCH 3/7] config: add the core.enablesha1dc setting Johannes Schindelin
2017-03-24 23:25 ` [PATCH 4/7] t0013: do not skip the entire file wholesale without DC_SHA1 Johannes Schindelin
2017-03-24 23:25 ` [PATCH 5/7] t0013: test DC_AND_OPENSSL_SHA1, too Johannes Schindelin
2017-03-24 23:28 ` [PATCH 6/7] mingw: enable DC_AND_OPENSSL_SHA1 by default Johannes Schindelin
2017-03-24 23:28 ` Johannes Schindelin [this message]
2017-03-25  6:37 ` [PATCH 0/7] PREVIEW: Introduce DC_AND_OPENSSL_SHA1 make flag Junio C Hamano
2017-03-25 16:58   ` Junio C Hamano
2017-03-26  6:18   ` Jeff King
2017-03-26 23:16     ` Junio C Hamano
2017-03-27  1:11       ` Jeff King
2017-03-27  6:07         ` Junio C Hamano
2017-03-27  7:09           ` Jeff King
2017-03-27 17:15             ` Junio C Hamano
2017-03-29 20:02   ` Johannes Schindelin
2017-03-30  0:31     ` Junio C Hamano
2017-04-18 11:30       ` Johannes Schindelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4c2e3b2abc84a4b4eb7b41b424f76defda550646.1490397869.git.johannes.schindelin@gmx.de \
    --to=johannes.schindelin@gmx.de \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).