git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: "brian m. carlson" <sandals@crustytoothpaste.net>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 3/3] index-pack: adjust default threading cap
Date: Mon, 24 Aug 2020 13:37:35 -0400	[thread overview]
Message-ID: <20200824173735.GA673908@coredump.intra.peff.net> (raw)
In-Reply-To: <20200822011607.GX8085@camp.crustytoothpaste.net>

On Sat, Aug 22, 2020 at 01:16:07AM +0000, brian m. carlson wrote:

> > +		if (nr_threads < 4)
> > +			; /* too few cores to consider capping */
> > +		else if (nr_threads < 6)
> > +			nr_threads = 3; /* historic cap */
> > +		else if (nr_threads < 40)
> > +			nr_threads /= 2;
> 
> I was going to ask if we could make the halving conditional on x86_64,
> but it turns out POWER and UltraSPARC also have SMT, so that doesn't
> make sense.  I expect that most users who have more than 6 "cores" are
> going to be on one of those systems or possibly ARM64, and since the
> performance penalty of using half as many cores isn't that significant,
> I'm not sure it's worth worrying about further.  This will be an
> improvement regardless.
> 
> Which is just a long way of saying, this patch seems fine to me.

OK, good. :) I agree there may be room for more improvement on those
systems. But lacking access to any, my goal was to make things better on
systems I _could_ test on, and not make things worse on other systems.
So I'd be very happy if people on other platforms (especially non-intel
ones) wanted to run:

  cd t/perf
  GIT_PERF_EXTRA=1 \
  GIT_PERF_LARGE_REPO=/path/to/clone/of/linux.git \
  ./p5302-pack-index.sh

and report the results.

I do have a slightly-old AMD machine with 4 cores (an A8-7600). Here's
what it says:

  5302.3: index-pack 0 threads                   447.67(436.62+6.57)
  5302.4: index-pack 1 threads                   450.80(441.26+7.20)
  5302.5: index-pack 2 threads                   265.62(459.56+7.30)
  5302.6: index-pack 4 threads                   177.06(477.56+8.22)
  5302.7: index-pack default number of threads   202.60(473.15+7.61)

So it does get better with 4 threads (but we continue to cap it at 3).
I wonder whether we should just do:

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 9721bf1ffe..d7453d0c09 100644
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -1809,7 +1809,7 @@ int cmd_index_pack(int argc, const char **argv, const char *prefix)
 		if (nr_threads < 4)
 			; /* too few cores to consider capping */
 		else if (nr_threads < 6)
-			nr_threads = 3; /* historic cap */
+			nr_threads = nr_threads;
 		else if (nr_threads < 40)
 			nr_threads /= 2;
 		else

That does probably make things slightly worse for a 6-core hyperthreaded
Intel machine. And it doesn't help an actual 8-core AMD machine at all.

-Peff

  reply	other threads:[~2020-08-24 17:37 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-21 17:51 [PATCH 0/3] index-pack threading defaults Jeff King
2020-08-21 17:53 ` [PATCH 1/3] p5302: disable thread-count parameter tests by default Jeff King
2020-08-21 17:54 ` [PATCH 2/3] p5302: count up to online-cpus for thread tests Jeff King
2020-08-21 17:58   ` Jeff King
2020-08-21 17:58 ` [PATCH 3/3] index-pack: adjust default threading cap Jeff King
2020-08-21 18:08   ` Eric Sunshine
2020-08-21 18:41     ` Jeff King
2020-08-22  1:16   ` brian m. carlson
2020-08-24 17:37     ` Jeff King [this message]
2020-08-24 17:55       ` Eric Sunshine
2020-08-21 18:44 ` [PATCH 0/3] index-pack threading defaults Jeff King
2020-08-21 18:59   ` Junio C Hamano
2020-08-21 19:14     ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200824173735.GA673908@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=sandals@crustytoothpaste.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).