git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff Hostetler <git@jeffhostetler.com>
To: Jeff King <peff@peff.net>,
	Johannes Schindelin <johannes.schindelin@gmx.de>
Cc: git@vger.kernel.org, Junio C Hamano <gitster@pobox.com>,
	Jeff Hostetler <jeffhost@microsoft.com>
Subject: Re: [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area
Date: Wed, 15 Feb 2017 09:27:53 -0500	[thread overview]
Message-ID: <16b1259c-4cdc-8f4d-db47-d724386a3d2b@jeffhostetler.com> (raw)
In-Reply-To: <20170214220332.753i4tgclm62er4f@sigill.intra.peff.net>



On 2/14/2017 5:03 PM, Jeff King wrote:
> On Tue, Feb 14, 2017 at 12:31:46PM +0100, Johannes Schindelin wrote:
>
>> On Windows, calls to memihash() and maintaining the istate.name_hash and
>> istate.dir_hash HashMaps take significant time on very large
>> repositories. This series of changes reduces the overall time taken for
>> various operations by reducing the number calls to memihash(), moving
>> some of them into multi-threaded code, and etc.
>>
>> Note: one commenter in https://github.com/git-for-windows/git/pull/964
>> pointed out that memihash() only handles ASCII correctly. That is true.
>> And fixing this is outside the purview of this patch series.
> Out of curiosity, do you have numbers? Bonus points if the speedup can
> be shown via a t/perf script.
>
> We have a read-cache perf-test already, but I suspect you'd want
> something more like "git status" or "ls-files -o" that calls into
> read_directory().

I have some informal numbers in a spreadsheet.  I was seeing
a 8-9% speed up on a status on my gigantic repo.

I'll try to put together a before/after perf-test to better demonstrate 
this.

Jeff


  reply	other threads:[~2017-02-15 14:27 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-14 11:31 [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area Johannes Schindelin
2017-02-14 11:31 ` [PATCH 1/5] name-hash: eliminate duplicate memihash call Johannes Schindelin
2017-02-14 11:32 ` [PATCH 2/5] hashmap: allow memihash computation to be continued Johannes Schindelin
2017-02-18  5:35   ` Junio C Hamano
2017-02-20 12:43     ` Johannes Schindelin
2017-02-20 20:27       ` Junio C Hamano
2017-02-14 11:32 ` [PATCH 3/5] name-hash: precompute hash values during preload-index Johannes Schindelin
2017-02-18  5:47   ` Junio C Hamano
2017-02-19  0:19     ` Jeff Hostetler
2017-02-19 21:45       ` Junio C Hamano
2017-02-14 11:32 ` [PATCH 4/5] name-hash: specify initial size for istate.dir_hash table Johannes Schindelin
2017-02-14 11:32 ` [PATCH 5/5] name-hash: remember previous dir_entry during lazy_init_name_hash Johannes Schindelin
2017-02-14 22:03 ` [PATCH 0/5] A series of performance enhancements in the memihash and name-cache area Jeff King
2017-02-15 14:27   ` Jeff Hostetler [this message]
2017-02-15 16:44     ` Jeff King
2017-02-18  5:56       ` Junio C Hamano
2017-02-19  0:02         ` Jeff Hostetler
2017-02-18  5:58     ` Junio C Hamano
2017-02-18  6:29       ` Jeff King
2017-02-18 20:48         ` Junio C Hamano
2017-02-18 23:52           ` Jeff Hostetler
2017-02-19 21:50             ` Junio C Hamano
2017-03-02 21:11     ` Junio C Hamano
2017-03-02 21:18       ` Jeff Hostetler
2017-03-02 21:40         ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=16b1259c-4cdc-8f4d-db47-d724386a3d2b@jeffhostetler.com \
    --to=git@jeffhostetler.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jeffhost@microsoft.com \
    --cc=johannes.schindelin@gmx.de \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).