git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / Atom feed
From: "Carlo Marcelo Arenas Belón" <carenas@gmail.com>
To: git@vger.kernel.org
Cc: frekui@gmail.com, michael.osipov@siemens.com,
	ibrahim.vanak@hpe.com, matheus.bernardino@usp.br,
	vleschuk@accesssoftek.com
Subject: [RFC PATCH 0/2] grep: make threading smarter
Date: Sun,  4 Aug 2019 14:15:07 -0700
Message-ID: <20190804211509.39229-1-carenas@gmail.com> (raw)

833e3df171 (pack-objects: Add runtime detection of online CPU's, 2008-02-22)
added the capability to check the number of online CPUs at runtime to do
better threading, so use that as well with grep.

Testing with a large (more than 4) number of cores and no grep.threads
configuration in real hardware encouraged, to confirm that no other
bottleneck is preventing the additional threads to improve performance.

If platform specific testing shows degradation (specially with HP-UX),
make sure that the right number of CPUs is reported by :

  $ ./t/helper/test-tool online-cpus
  4

There are additional cleanups possible in the grep code but had left it
out of this RFC to avoid confusion and make the change in patch 2 as
straight forward as possible.

There is also a chance that the online_cpus() function will be updated
as it predates POSIX and might be associated with one known performance
issue in HP-UX[1]

Lastly the performance numbers point to deficiencies in kwset and the
compat/regex code that will need to be addressed independently.

Carlo Marcelo Arenas Belón (2):
  p7810: add more grep performance relevant cases
  grep: make default number of threads reflect runtime

 Documentation/git-grep.txt |  2 +-
 builtin/grep.c             |  3 +--
 t/perf/p7810-grep.sh       | 25 +++++++++++++++++++++++--
 3 files changed, 25 insertions(+), 5 deletions(-)

[1] https://public-inbox.org/git/TU4PR8401MB121664A8A588D799803F1E84E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM/
-- 
2.23.0.rc1

             reply	other threads:[~2019-08-04 21:15 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-04 21:15 Carlo Marcelo Arenas Belón [this message]
2019-08-04 21:15 ` [RFC PATCH 1/2] p7810: add more grep performance relevant cases Carlo Marcelo Arenas Belón
2019-08-04 21:15 ` [RFC PATCH 2/2] grep: make default number of threads reflect runtime Carlo Marcelo Arenas Belón
2019-08-05 21:28   ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190804211509.39229-1-carenas@gmail.com \
    --to=carenas@gmail.com \
    --cc=frekui@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=ibrahim.vanak@hpe.com \
    --cc=matheus.bernardino@usp.br \
    --cc=michael.osipov@siemens.com \
    --cc=vleschuk@accesssoftek.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 git git/ https://public-inbox.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.io/gmane.comp.version-control.git
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for the project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git