git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>, "Jeff King" <peff@peff.net>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH v3 0/6] Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets
Date: Tue, 23 Feb 2021 12:41:26 +0100	[thread overview]
Message-ID: <20210223114132.24345-1-avarab@gmail.com> (raw)
In-Reply-To: <20210201111715.10200-1-avarab@gmail.com>

Addresses feedback on v2:
https://lore.kernel.org/git/20210201111715.10200-1-avarab@gmail.com/

Changes:

 - Added .PHONY targets as appropriate

 - Instead of removing fuzz-objs from "all" we now run it in the CI
   build instead. I think this accomplishes the goal of avoiding
   bitrot without needlessly compiling them on every build of git.

As Jeff points out in
https://lore.kernel.org/git/YBuc5iOCCHk4fPqs@coredump.intra.peff.net/
the use-case for having "{program-xdiff,test,git}-objs & objects"
targets is a bit harder to justify.

I still think they're useful, particularly for testing on e.g. slow
single-core VMs or other test setups (I use the GCC farm) where I know
I just want to compile e.g. "test" objects, and compiling one of them
takes 1-2 seconds.

It's an easy enough patch to carry, and now with 6/6 we even have an
in-tree consumer of one of them.

Ævar Arnfjörð Bjarmason (6):
  Makefile: guard against TEST_OBJS in the environment
  Makefile: split up long OBJECTS line
  Makefile: sort OBJECTS assignment for subsequent change
  Makefile: split OBJECTS into OBJECTS and GIT_OBJS
  Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets
  Makefile: build "$(FUZZ_OBJS)" in CI, not under "all"

 Makefile                  | 38 ++++++++++++++++++++++++++++----------
 ci/run-build-and-tests.sh |  1 +
 2 files changed, 29 insertions(+), 10 deletions(-)

Range-diff:
2:  a50b68fe195 = 1:  cf6d71dcf5a Makefile: guard against TEST_OBJS in the environment
3:  53656000ebe = 2:  ad7ac896c09 Makefile: split up long OBJECTS line
4:  d956624baea = 3:  575b2ab8e9c Makefile: sort OBJECTS assignment for subsequent change
5:  500ace9cfb4 = 4:  7fdaeb3616b Makefile: split OBJECTS into OBJECTS and GIT_OBJS
6:  8f7ce09e9bd ! 5:  765cf20c58c Makefile: add {program,xdiff,test,git}-objs & objects targets
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    Makefile: add {program,xdiff,test,git}-objs & objects targets
    +    Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets
     
         Add targets to compile the various *.o files we declared in commonly
         used *_OBJS variables. This is useful for debugging purposes, to
    @@ Commit message
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Makefile ##
    +@@ Makefile: ETAGS_TARGET = TAGS
    + FUZZ_OBJS += fuzz-commit-graph.o
    + FUZZ_OBJS += fuzz-pack-headers.o
    + FUZZ_OBJS += fuzz-pack-idx.o
    ++.PHONY: fuzz-objs
    ++fuzz-objs: $(FUZZ_OBJS)
    + 
    + # Always build fuzz objects even if not testing, to prevent bit-rot.
    + all:: $(FUZZ_OBJS)
     @@ Makefile: PROGRAM_OBJS += http-backend.o
      PROGRAM_OBJS += imap-send.o
      PROGRAM_OBJS += sh-i18n--envsubst.o
      PROGRAM_OBJS += shell.o
    ++.PHONY: program-objs
     +program-objs: $(PROGRAM_OBJS)
      
      # Binary suffix, set to .exe for Windows builds
    @@ Makefile: XDIFF_OBJS += xdiff/xmerge.o
      XDIFF_OBJS += xdiff/xpatience.o
      XDIFF_OBJS += xdiff/xprepare.o
      XDIFF_OBJS += xdiff/xutils.o
    ++.PHONY: xdiff-objs
     +xdiff-objs: $(XDIFF_OBJS)
      
      TEST_OBJS := $(patsubst %$X,%.o,$(TEST_PROGRAMS)) $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))
    ++.PHONY: test-objs
     +test-objs: $(TEST_OBJS)
      
      GIT_OBJS += $(LIB_OBJS)
      GIT_OBJS += $(BUILTIN_OBJS)
      GIT_OBJS += common-main.o
      GIT_OBJS += git.o
    ++.PHONY: git-objs
     +git-objs: $(GIT_OBJS)
      
      OBJECTS += $(GIT_OBJS)
    @@ Makefile: OBJECTS += $(FUZZ_OBJS)
      ifndef NO_CURL
      	OBJECTS += http.o http-walker.o remote-curl.o
      endif
    ++.PHONY: objects
     +objects: $(OBJECTS)
      
      dep_files := $(foreach f,$(OBJECTS),$(dir $f).depend/$(notdir $f).d)
1:  20ec032b390 ! 6:  bfedec4e5b4 Makefile: remove "all" on "$(FUZZ_OBJS)"
    @@ Metadata
     Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Commit message ##
    -    Makefile: remove "all" on "$(FUZZ_OBJS)"
    +    Makefile: build "$(FUZZ_OBJS)" in CI, not under "all"
     
    -    Adding this as a dependency was intentionally done in
    +    Adding $(FUZZ_OBJS) as a dependency on "all" was intentionally done in
         5e472150800 (fuzz: add basic fuzz testing target., 2018-10-12).
     
    -    I don't see why we need to prevent bitrot here under "all" for these
    -    in particular, but not e.g. contrib/credential/**/*.c
    +    Rather than needlessly build these objects which aren't required for
    +    the build every time we make "all", let's instead move them to be
    +    built by the CI jobs.
     
    -    In any case, these files are rather trivial and from their commit log
    -    it seems the fuzz-all target is run by a few people already.
    +    The goal is to make sure that we don't inadvertently break these, we
    +    can accomplish that goal by building them in CI, rather than slowing
    +    down every build of git for everyone everywhere.
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
     
      ## Makefile ##
    -@@ Makefile: FUZZ_OBJS += fuzz-commit-graph.o
    - FUZZ_OBJS += fuzz-pack-headers.o
    - FUZZ_OBJS += fuzz-pack-idx.o
    +@@ Makefile: FUZZ_OBJS += fuzz-pack-idx.o
    + .PHONY: fuzz-objs
    + fuzz-objs: $(FUZZ_OBJS)
      
     -# Always build fuzz objects even if not testing, to prevent bit-rot.
     -all:: $(FUZZ_OBJS)
    @@ Makefile: FUZZ_OBJS += fuzz-commit-graph.o
      FUZZ_PROGRAMS += $(patsubst %.o,%,$(FUZZ_OBJS))
      
      # Empty...
    -@@ Makefile: $(FUZZ_PROGRAMS): all
    +@@ Makefile: FUZZ_CXXFLAGS ?= $(CFLAGS)
    + 
    + .PHONY: fuzz-all
    + 
    +-$(FUZZ_PROGRAMS): all
    ++$(FUZZ_PROGRAMS): all fuzz-objs
      	$(QUIET_LINK)$(CXX) $(FUZZ_CXXFLAGS) $(LIB_OBJS) $(BUILTIN_OBJS) \
      		$(XDIFF_OBJS) $(EXTLIBS) git.o $@.o $(LIB_FUZZING_ENGINE) -o $@
      
     -fuzz-all: $(FUZZ_PROGRAMS)
     +fuzz-all: $(FUZZ_PROGRAMS) $(FUZZ_OBJS)
    +
    + ## ci/run-build-and-tests.sh ##
    +@@ ci/run-build-and-tests.sh: windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";;
    + *) ln -s "$cache_dir/.prove" t/.prove;;
    + esac
    + 
    ++make fuzz-objs
    + make
    + case "$jobname" in
    + linux-gcc)
-- 
2.30.0.284.gd98b1dd5eaa7


  parent reply	other threads:[~2021-02-23 11:44 UTC|newest]

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-26 16:07 [PATCH 0/4] Makefile: micro-optimize light non-test builds Ævar Arnfjörð Bjarmason
2021-01-26 16:07 ` [PATCH 1/4] Makefile: refactor assignment for subsequent change Ævar Arnfjörð Bjarmason
2021-01-27  1:29   ` Junio C Hamano
2021-01-26 16:07 ` [PATCH 2/4] Makefile: refactor " Ævar Arnfjörð Bjarmason
2021-01-26 16:07 ` [PATCH 3/4] Makefile: add a NO_TEST_TOOLS flag Ævar Arnfjörð Bjarmason
2021-01-26 16:07 ` [PATCH 4/4] Makefile: add a NO_{INSTALL_,}SCRIPT_FALLBACKS target Ævar Arnfjörð Bjarmason
2021-01-26 21:16 ` [PATCH 0/4] Makefile: micro-optimize light non-test builds Jeff King
2021-01-27  1:38   ` Junio C Hamano
2021-01-27  4:34     ` Jeff King
2021-01-27  6:07       ` Junio C Hamano
2021-01-28 18:23   ` [PATCH 0/6] Makefile: add {program,xdiff,test,git}-objs & objects targets Ævar Arnfjörð Bjarmason
2021-02-01 11:17     ` [PATCH v2 " Ævar Arnfjörð Bjarmason
2021-02-03  1:11       ` Junio C Hamano
2021-02-04  7:06         ` Jeff King
2021-02-04 17:49           ` Junio C Hamano
2021-02-23 11:41       ` Ævar Arnfjörð Bjarmason [this message]
2021-02-23 17:57         ` [PATCH v3 0/6] Makefile: add {program,xdiff,test,git,fuzz}-objs " Junio C Hamano
2021-02-23 18:31         ` Jeff King
2021-02-23 11:41       ` [PATCH v3 1/6] Makefile: guard against TEST_OBJS in the environment Ævar Arnfjörð Bjarmason
2021-02-23 11:41       ` [PATCH v3 2/6] Makefile: split up long OBJECTS line Ævar Arnfjörð Bjarmason
2021-02-23 11:41       ` [PATCH v3 3/6] Makefile: sort OBJECTS assignment for subsequent change Ævar Arnfjörð Bjarmason
2021-02-23 11:41       ` [PATCH v3 4/6] Makefile: split OBJECTS into OBJECTS and GIT_OBJS Ævar Arnfjörð Bjarmason
2021-02-23 11:41       ` [PATCH v3 5/6] Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets Ævar Arnfjörð Bjarmason
2021-02-23 11:41       ` [PATCH v3 6/6] Makefile: build "$(FUZZ_OBJS)" in CI, not under "all" Ævar Arnfjörð Bjarmason
2021-02-23 18:28         ` Jeff King
2021-02-23 19:19           ` Junio C Hamano
2021-02-28 20:13           ` Ævar Arnfjörð Bjarmason
2021-03-01  9:39             ` Jeff King
2021-02-01 11:17     ` [PATCH v2 1/6] Makefile: remove "all" on "$(FUZZ_OBJS)" Ævar Arnfjörð Bjarmason
2021-02-04  6:51       ` Jeff King
2021-02-01 11:17     ` [PATCH v2 2/6] Makefile: guard against TEST_OBJS in the environment Ævar Arnfjörð Bjarmason
2021-02-01 11:17     ` [PATCH v2 3/6] Makefile: split up long OBJECTS line Ævar Arnfjörð Bjarmason
2021-02-01 11:17     ` [PATCH v2 4/6] Makefile: sort OBJECTS assignment for subsequent change Ævar Arnfjörð Bjarmason
2021-02-01 11:17     ` [PATCH v2 5/6] Makefile: split OBJECTS into OBJECTS and GIT_OBJS Ævar Arnfjörð Bjarmason
2021-02-01 11:17     ` [PATCH v2 6/6] Makefile: add {program,xdiff,test,git}-objs & objects targets Ævar Arnfjörð Bjarmason
2021-02-01 22:30       ` Junio C Hamano
2021-01-28 18:23   ` [PATCH 1/6] Makefile: remove "all" on "$(FUZZ_OBJS)" Ævar Arnfjörð Bjarmason
2021-01-28 18:23   ` [PATCH 2/6] Makefile: guard against TEST_OBJS in the environment Ævar Arnfjörð Bjarmason
2021-01-29  7:49     ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210223114132.24345-1-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).