git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / Atom feed
From: "Carlo Marcelo Arenas Belón" <carenas@gmail.com>
To: git@vger.kernel.org
Cc: frekui@gmail.com, michael.osipov@siemens.com,
	ibrahim.vanak@hpe.com, matheus.bernardino@usp.br,
	vleschuk@accesssoftek.com
Subject: [RFC PATCH 2/2] grep: make default number of threads reflect runtime
Date: Sun,  4 Aug 2019 14:15:09 -0700
Message-ID: <20190804211509.39229-3-carenas@gmail.com> (raw)
In-Reply-To: <20190804211509.39229-1-carenas@gmail.com>

5b594f457a (Threaded grep, 2010-01-25) added a hardcoded number of
threads(8) to use in grep and 89f09dd34e (grep: add --threads=<num>
option and grep.threads configuration, 2015-12-15) made it configurable
through a knob as a workaround for systems where that default was not
effective.

Use instead the industry standard of 2x number of CPUs (to allow for
IO wait) for the default.

Using Debian 10 amd64 in a 2 CPU VirtualBox running in macOS 10.14.6
and that might had been representative of the original author environment
shows an overall performance improvement by avoiding thread trashing:

Test                                                origin/maint        HEAD
-----------------------------------------------------------------------------------------------
7810.1: grep worktree, fixed regex (no match)       0.52(0.43+0.49)     0.50(0.44+0.46) -3.8%
7810.2: grep worktree, fixed regex (common)         0.94(1.20+0.50)     0.91(1.24+0.44) -3.2%
7810.3: grep -I, fixed non binary regex (common)    0.98(1.24+0.51)     0.94(1.30+0.44) -4.1%
7810.4: grep -i, fixed caseless regex (common)      0.97(1.31+0.45)     0.93(1.18+0.56) -4.1%
7810.5: grep --no-index, fixed regex (common)       1.02(1.28+0.50)     0.97(1.14+0.59) -4.9%
7810.6: grep worktree, simple regex (common)        0.77(0.96+0.45)     0.73(0.88+0.48) -5.2%
7810.7: grep -I, simple non binary regex (common)   0.78(0.96+0.48)     0.73(0.94+0.43) -6.4%
7810.8: grep -i, simple caseless regex (common)     0.87(1.11+0.48)     0.82(1.16+0.38) -5.7%
7810.9: grep worktree, expensive regex              10.37(19.67+0.76)   10.20(19.46+0.76) -1.6%
7810.10: grep --cached, fixed regex                 4.48(4.37+0.10)     4.63(4.54+0.09) +3.3%
7810.11: grep --cached, expensive regex             23.74(23.61+0.11)   23.39(23.28+0.09) -1.5%

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
---
 Documentation/git-grep.txt | 2 +-
 builtin/grep.c             | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index 2d27969057..5d72e03b2e 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -60,7 +60,7 @@ grep.extendedRegexp::
 
 grep.threads::
 	Number of grep worker threads to use.  If unset (or set to 0),
-	8 threads are used by default (for now).
+	2 threads per core are used by default.
 
 grep.fullName::
 	If set to true, enable `--full-name` option by default.
diff --git a/builtin/grep.c b/builtin/grep.c
index 580fd38f41..0ed8da30f8 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -32,7 +32,6 @@ static char const * const grep_usage[] = {
 
 static int recurse_submodules;
 
-#define GREP_NUM_THREADS_DEFAULT 8
 static int num_threads;
 
 static pthread_t *threads;
@@ -1068,7 +1067,7 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 	} else if (num_threads < 0)
 		die(_("invalid number of threads specified (%d)"), num_threads);
 	else if (num_threads == 0)
-		num_threads = HAVE_THREADS ? GREP_NUM_THREADS_DEFAULT : 1;
+		num_threads = HAVE_THREADS ? online_cpus() * 2 : 1;
 
 	if (num_threads > 1) {
 		if (!HAVE_THREADS)
-- 
2.23.0.rc1


  parent reply	other threads:[~2019-08-04 21:15 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-04 21:15 [RFC PATCH 0/2] grep: make threading smarter Carlo Marcelo Arenas Belón
2019-08-04 21:15 ` [RFC PATCH 1/2] p7810: add more grep performance relevant cases Carlo Marcelo Arenas Belón
2019-08-04 21:15 ` Carlo Marcelo Arenas Belón [this message]
2019-08-05 21:28   ` [RFC PATCH 2/2] grep: make default number of threads reflect runtime Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190804211509.39229-3-carenas@gmail.com \
    --to=carenas@gmail.com \
    --cc=frekui@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=ibrahim.vanak@hpe.com \
    --cc=matheus.bernardino@usp.br \
    --cc=michael.osipov@siemens.com \
    --cc=vleschuk@accesssoftek.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git