git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Phillip Wood" <phillip.wood123@gmail.com>,
	"Jeff King" <peff@peff.net>, "Dan Jacques" <dnj@google.com>,
	"Eric Wong" <e@80x24.org>, "Jonathan Nieder" <jrnieder@gmail.com>,
	"Mike Hommey" <mh@glandium.org>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v3 22/23] Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES)
Date: Tue, 16 Nov 2021 13:00:22 +0100	[thread overview]
Message-ID: <patch-v3-22.23-88cfc946b37-20211116T114334Z-avarab@gmail.com> (raw)
In-Reply-To: <cover-v3-00.23-00000000000-20211116T114334Z-avarab@gmail.com>

Combine the definitions of $(FIND_SOURCE_FILES) and $(LIB_H) to speed
up the Makefile, as these are the two main expensive $(shell) commands
that we execute unconditionally.

When see what was in $(FOUND_SOURCE_FILES) that wasn't in $(LIB_H) via
the ad-hoc test of:

    $(error $(filter-out $(LIB_H),$(filter %.h,$(ALL_SOURCE_FILES))))
    $(error $(filter-out $(ALL_SOURCE_FILES),$(filter %.h,$(LIB_H))))

We'll get, respectively:

    Makefile:850: *** t/helper/test-tool.h.  Stop.
    Makefile:850: *** .  Stop.

I.e. we only had a discrepancy when it came to
t/helper/test-tool.h. In terms of correctness this was broken before,
but now works:

    $ make t/helper/test-tool.hco
        HDR t/helper/test-tool.h

This speeds things up a lot:

    $ git -c hyperfine.hook.setup= hyperfine -L rev HEAD~1,HEAD~0 -s 'make -j8 all NO_TCLTK=Y' 'make NO_TCLTK=Y' --war
    mup 10 -M 10
    Benchmark 1: make NO_TCLTK=Y' in 'HEAD~1
      Time (mean ± σ):      99.5 ms ±   0.5 ms    [User: 83.4 ms, System: 20.7 ms]
      Range (min … max):    98.8 ms … 100.2 ms    10 runs

    Benchmark 2: make NO_TCLTK=Y' in 'HEAD~0
      Time (mean ± σ):      69.4 ms ±   0.5 ms    [User: 58.8 ms, System: 14.2 ms]
      Range (min … max):    68.9 ms …  70.3 ms    10 runs

    Summary
      'make NO_TCLTK=Y' in 'HEAD~0' ran
        1.43 ± 0.01 times faster than 'make NO_TCLTK=Y' in 'HEAD~1'

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Makefile | 54 ++++++++++++++++++++++++++----------------------------
 1 file changed, 26 insertions(+), 28 deletions(-)

diff --git a/Makefile b/Makefile
index bae20eb1b40..1041efb6e5e 100644
--- a/Makefile
+++ b/Makefile
@@ -821,12 +821,33 @@ GENERATED_H += hook-list.h
 .PHONY: generated-hdrs
 generated-hdrs: $(GENERATED_H)
 
-LIB_H := $(sort $(patsubst ./%,%,$(shell git ls-files '*.h' ':!t/' ':!Documentation/' 2>/dev/null || \
+## Exhaustive lists of our source files, either dynamically generated,
+## or hardcoded.
+SOURCES_CMD = ( \
+	git ls-files \
+		'*.[hcS]' \
+		'*.sh' \
+		':!*[tp][0-9][0-9][0-9][0-9]*' \
+		':!contrib' \
+		2>/dev/null || \
 	$(FIND) . \
-	-name .git -prune -o \
-	-name t -prune -o \
-	-name Documentation -prune -o \
-	-name '*.h' -print)))
+		\( -name .git -type d -prune \) \
+		-o \( -name '[tp][0-9][0-9][0-9][0-9]*' -prune \) \
+		-o \( -name contrib -type d -prune \) \
+		-o \( -name build -type d -prune \) \
+		-o \( -name 'trash*' -type d -prune \) \
+		-o \( -name '*.[hcS]' -type f -print \) \
+		-o \( -name '*.sh' -type f -print \) \
+		| sed -e 's|^\./||' \
+	)
+FOUND_SOURCE_FILES := $(shell $(SOURCES_CMD))
+
+FOUND_C_SOURCES = $(filter %.c,$(FOUND_SOURCE_FILES))
+FOUND_H_SOURCES = $(filter %.h,$(FOUND_SOURCE_FILES))
+
+COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))
+
+LIB_H = $(FOUND_H_SOURCES)
 
 LIB_OBJS += abspath.o
 LIB_OBJS += add-interactive.o
@@ -2661,26 +2682,6 @@ perl/build/man/man3/Git.3pm: perl/Git.pm
 	$(call mkdir_p_parent_template)
 	$(QUIET_GEN)pod2man $< $@
 
-FIND_SOURCE_FILES = ( \
-	git ls-files \
-		'*.[hcS]' \
-		'*.sh' \
-		':!*[tp][0-9][0-9][0-9][0-9]*' \
-		':!contrib' \
-		2>/dev/null || \
-	$(FIND) . \
-		\( -name .git -type d -prune \) \
-		-o \( -name '[tp][0-9][0-9][0-9][0-9]*' -prune \) \
-		-o \( -name contrib -type d -prune \) \
-		-o \( -name build -type d -prune \) \
-		-o \( -name 'trash*' -type d -prune \) \
-		-o \( -name '*.[hcS]' -type f -print \) \
-		-o \( -name '*.sh' -type f -print \) \
-		| sed -e 's|^\./||' \
-	)
-
-FOUND_SOURCE_FILES = $(shell $(FIND_SOURCE_FILES))
-
 $(ETAGS_TARGET): $(FOUND_SOURCE_FILES)
 	$(QUIET_GEN)$(RM) $@+ && \
 	echo $(FOUND_SOURCE_FILES) | xargs etags -a -o $@+ && \
@@ -2860,9 +2861,6 @@ check: $(GENERATED_H)
 		exit 1; \
 	fi
 
-FOUND_C_SOURCES = $(filter %.c,$(FOUND_SOURCE_FILES))
-COCCI_SOURCES = $(filter-out $(THIRD_PARTY_SOURCES),$(FOUND_C_SOURCES))
-
 %.cocci.patch: %.cocci $(COCCI_SOURCES)
 	$(QUIET_SPATCH) \
 	if test $(SPATCH_BATCH_SIZE) = 0; then \
-- 
2.34.0.795.g1e9501ab396


  parent reply	other threads:[~2021-11-16 12:06 UTC|newest]

Thread overview: 107+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-06 21:03 [PATCH 00/16] Makefiles: dependency correctness & speedup Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 01/16] Makefile: don't set up "perl/build" rules under NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 02/16] Makefile: clean perl/build/ even with NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-09 23:10   ` Junio C Hamano
2021-11-06 21:03 ` [PATCH 03/16] Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 04/16] Makefile: move Perl-only variable assignments under !NO_PERL Ævar Arnfjörð Bjarmason
2021-11-09 23:22   ` Junio C Hamano
2021-11-06 21:03 ` [PATCH 05/16] Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph Ævar Arnfjörð Bjarmason
2021-11-07  1:51   ` Eric Sunshine
2021-11-06 21:03 ` [PATCH 06/16] Makefile: don't have Perl over-depend on GIT-BUILD-OPTIONS Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 07/16] Makefile: create a GIT-PYTHON-DEFINES, like "PERL" Ævar Arnfjörð Bjarmason
2021-11-07  1:55   ` Eric Sunshine
2021-11-06 21:03 ` [PATCH 08/16] Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 09/16] Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 10/16] Makefile: move $(comma), $(empty) and $(space) to shared.mak Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 11/16] Makefile: re-add and use the "shellquote" macros Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 12/16] Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...} Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 13/16] Makefile: add "$(QUIET)" boilerplate to shared.mak Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 14/16] Makefile: use $(wspfx) for $(QUIET...) in shared.mak Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 15/16] Makefiles: add and use wildcard "mkdir -p" template Ævar Arnfjörð Bjarmason
2021-11-06 21:03 ` [PATCH 16/16] Makefile: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2021-11-12 21:48 ` [PATCH v2 00/18] Makefiles: dependency correctness & speedup Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 01/18] Makefile: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 02/18] Makefile: don't set up "perl/build" rules under NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 03/18] Makefile: use "=" not ":=" for po/* and perl/* Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 04/18] Makefile: clean perl/build/ even with NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 05/18] Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 06/18] Makefile: guard Perl-only variable assignments Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 07/18] Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL" Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 08/18] Makefile: adjust Perl-related comments & whitespace Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 09/18] Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 10/18] Makefile: create a GIT-PYTHON-DEFINES, like "PERL" Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 11/18] Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 12/18] Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 13/18] Makefile: move $(comma), $(empty) and $(space) to shared.mak Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 14/18] Makefile: re-add and use the "shellquote" macros Ævar Arnfjörð Bjarmason
2021-11-12 21:48   ` [PATCH v2 15/18] Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...} Ævar Arnfjörð Bjarmason
2021-11-12 21:49   ` [PATCH v2 16/18] Makefile: add "$(QUIET)" boilerplate to shared.mak Ævar Arnfjörð Bjarmason
2021-11-12 21:49   ` [PATCH v2 17/18] Makefile: use $(wspfx) for $(QUIET...) in shared.mak Ævar Arnfjörð Bjarmason
2021-11-12 21:49   ` [PATCH v2 18/18] Makefiles: add and use wildcard "mkdir -p" template Ævar Arnfjörð Bjarmason
2021-11-16 12:00   ` [PATCH v3 00/23] Makefile: dependency fixes, make noop runtime ~1.4x faster Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 01/23] Makefile: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 02/23] Makefile: don't set up "perl/build" rules under NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 03/23] Makefile: use "=" not ":=" for po/* and perl/* Ævar Arnfjörð Bjarmason
2021-11-17  1:57       ` Mike Hommey
2021-11-16 12:00     ` [PATCH v3 04/23] Makefile: clean perl/build/ even with NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 05/23] Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR Ævar Arnfjörð Bjarmason
2021-11-17  2:01       ` Mike Hommey
2021-11-17  9:20         ` Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 06/23] Makefile: guard Perl-only variable assignments Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 07/23] Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL" Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 08/23] Makefile: adjust Perl-related comments & whitespace Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 09/23] Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 10/23] Makefile: create a GIT-PYTHON-DEFINES, like "PERL" Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 11/23] Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 12/23] Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 13/23] Makefile: move $(comma), $(empty) and $(space) to shared.mak Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 14/23] Makefile: re-add and use the "shellquote" macros Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 15/23] Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...} Ævar Arnfjörð Bjarmason
2021-11-17  2:19       ` Mike Hommey
2021-11-17  9:24         ` Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 16/23] Makefile: add "$(QUIET)" boilerplate to shared.mak Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 17/23] Makefile: use $(wspfx) for $(QUIET...) in shared.mak Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 18/23] Makefiles: add and use wildcard "mkdir -p" template Ævar Arnfjörð Bjarmason
2021-11-17  2:51       ` Mike Hommey
2021-11-17  9:26         ` Ævar Arnfjörð Bjarmason
2021-11-17  9:39           ` Mike Hommey
2021-11-17 11:52             ` Ævar Arnfjörð Bjarmason
2021-11-18  0:00               ` Mike Hommey
2021-11-18 13:05                 ` Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 19/23] Makefile: correct the dependency graph of hook-list.h Ævar Arnfjörð Bjarmason
2021-11-17  2:52       ` Mike Hommey
2021-11-16 12:00     ` [PATCH v3 20/23] Makefile: use $(file) I/O instead of "FORCE" when possible Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` [PATCH v3 21/23] Makefile: disable GNU make built-in wildcard rules Ævar Arnfjörð Bjarmason
2021-11-17  3:00       ` Mike Hommey
2021-11-17  9:47         ` Ævar Arnfjörð Bjarmason
2021-11-16 12:00     ` Ævar Arnfjörð Bjarmason [this message]
2021-11-16 12:00     ` [PATCH v3 23/23] Makefile: move ".SUFFIXES" rule to shared.mak Ævar Arnfjörð Bjarmason
2021-11-17 10:19     ` [PATCH v4 00/23] Makefile: dependency fixes, make noop runtime ~1.4x faster Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 01/23] Makefile: don't invoke msgfmt with --statistics Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 02/23] Makefile: don't set up "perl/build" rules under NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 03/23] Makefile: use "=" not ":=" for po/* and perl/* Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 04/23] Makefile: clean perl/build/ even with NO_PERL=Y Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 05/23] Makefile: remove "mv $@ $@+" dance redundant to .DELETE_ON_ERROR Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 06/23] Makefile: guard Perl-only variable assignments Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 07/23] Makefile: change "ifndef NO_PERL" to "ifdef NO_PERL" Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 08/23] Makefile: adjust Perl-related comments & whitespace Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 09/23] Makefile: correct "GIT-PERL-{DEFINES,HEADER}" dependency graph Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 10/23] Makefile: create a GIT-PYTHON-DEFINES, like "PERL" Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 11/23] Makefile: stop needing @@GIT_VERSION@@ in *.perl scripts Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 12/23] Makefiles: add "shared.mak", move ".DELETE_ON_ERROR" to it Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 13/23] Makefile: move $(comma), $(empty) and $(space) to shared.mak Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 14/23] Makefile: re-add and use the "shellquote" macros Ævar Arnfjörð Bjarmason
2021-12-13 23:09         ` Junio C Hamano
2021-12-14  0:31           ` Junio C Hamano
2021-11-17 10:20       ` [PATCH v4 15/23] Makefile: add a "TRACK_template" for GIT-*{FLAGS,DEFINES,...} Ævar Arnfjörð Bjarmason
2021-11-19  6:45         ` Junio C Hamano
2021-11-19  6:56           ` Junio C Hamano
2021-11-19  7:30             ` Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 16/23] Makefile: add "$(QUIET)" boilerplate to shared.mak Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 17/23] Makefile: use $(wspfx) for $(QUIET...) in shared.mak Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 18/23] Makefiles: add and use wildcard "mkdir -p" template Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 19/23] Makefile: correct the dependency graph of hook-list.h Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 20/23] Makefile: use $(file) I/O instead of "FORCE" when possible Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 21/23] Makefile: disable GNU make built-in wildcard rules Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 22/23] Makefile: define $(LIB_H) in terms of $(FIND_SOURCE_FILES) Ævar Arnfjörð Bjarmason
2021-11-17 10:20       ` [PATCH v4 23/23] Makefile: move ".SUFFIXES" rule to shared.mak Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=patch-v3-22.23-88cfc946b37-20211116T114334Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=dnj@google.com \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jrnieder@gmail.com \
    --cc=mh@glandium.org \
    --cc=peff@peff.net \
    --cc=phillip.wood123@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).