Re: Why the Makefile is so eager to re-build & re-link

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Fri, Jun 25 2021, Ævar Arnfjörð Bjarmason wrote:

Just one small thing I forgot to note:

> On Thu, Jun 24 2021, Johannes Sixt wrote:
>> I have two patches that speed up generate-cmdlist.sh a bit:
> [...]
> Interesting, but I think rather than micro-optimizing the O(n) loop it
> makes more sense to turn it into a series of O(1) in -j parallel,
> i.e. actually use the make dependency graph for this as I suggested in:
> https://lore.kernel.org/git/87wnqiyejg.fsf@evledraar.gmail.com/
> [...]
> diff --git a/.gitignore b/.gitignore
> index 311841f9bed..9b365395496 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -13,6 +13,7 @@
>  /GIT-USER-AGENT
>  /GIT-VERSION-FILE
>  /bin-wrappers/
> +/build/
>  /git
>  /git-add
>  /git-add--interactive
> diff --git a/Makefile b/Makefile
> index c3565fc0f8f..5e845bd0f69 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -2231,12 +2231,30 @@ config-list.h: Documentation/*config.txt Documentation/config/*.txt
>  	$(QUIET_GEN)$(SHELL_PATH) ./generate-configlist.sh \
>  		>$@+ && mv $@+ $@
>  
> -command-list.h: generate-cmdlist.sh command-list.txt
> +build/Documentation:
> +	$(QUIET_GEN)mkdir -p build/Documentation
> +.PRECIOUS: build/Documentation/git%.txt.cmdlist.in
> +build/Documentation/git%.txt.cmdlist.in: Documentation/git%.txt
> +	$(QUIET_GEN)if ! grep -w $(patsubst build/Documentation/%.txt.cmdlist.in,%,$@) command-list.txt >$@; \
> +	then \
> +		# For e.g. git-init-db, which has a *.txt file, but no \
> +		# command-list.h entry \
> +		>$@; \
> +	fi
> +build/Documentation/git%.txt.cmdlist: build/Documentation/git%.txt.cmdlist.in
> +	$(QUIET_GEN)./generate-cmdlist.sh --tail $< >$@
>  
> -command-list.h: $(wildcard Documentation/git*.txt)
> -	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh \
> +COMMAND_LIST_H_FILES = $(filter-out Documentation/git.txt,$(wildcard Documentation/git*.txt))
> +
> +COMMAND_LIST_GEN = $(patsubst Documentation/%.txt,build/Documentation/%.txt.cmdlist,$(COMMAND_LIST_H_FILES))
> +command-list.h: build/Documentation generate-cmdlist.sh command-list.txt $(COMMAND_LIST_GEN)
> +	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh --header \
>  		$(patsubst %,--exclude-program %,$(EXCLUDED_PROGRAMS)) \

I left this --exclude-program code (added in 724d63569fe (help -a: do
not list commands that are excluded from the build, 2019-04-18)) in just
because I couldn't be bothered to rewrite it for this demo. But of
course that's also something that becomes much easier once we do this in
a Makefile-native way, we'd just $(filter) that list out ourselves, so
no need to have some shellscript deal with it.

That's also true on "master" right now, 724d63569fe doesn't discuss why
it went for the shellscript approach. I suspect it's just that the
author thought it was easier than dealing with some combination of
$(filter) and (a possibly nested) $(pathsubst).

> -		command-list.txt >$@+ && mv $@+ $@
> +		command-list.txt >$@+ && \
> +	echo "static struct cmdname_help command_list[] = {" >>$@+ && \
> +	cat build/Documentation/*.txt.cmdlist >>$@+ && \
> +	echo "};" >>$@+ && \
> +	mv $@+ $@