Re: [PATCH] Makefile: make COMPUTE_HEADER_DEPENDENCIES=auto work with DEVOPTS=pedantic

[Jeff King <peff@peff.net>]

On Thu, Sep 23, 2021 at 12:40:04AM +0200, Ævar Arnfjörð Bjarmason wrote:

> > It's a shame we can't just try to do the _real_ compiles using the
> > auto-dependency stuff, and then fall back if they fail. But I think
> > there's a chicken-and-egg problem there with "make" doing real work, and
> > figuring out the dependencies to do real work.
> 
> Isn't this just a chicken & egg problem because we've made it a chicken
> & egg problem? I.e. this WIP hack seems to work for me to avoid it:

Possibly, but...

> -$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
> +ifeq ($(COMPUTE_HEADER_DEPENDENCIES),yes)
> +$(C_DEP_OBJ): $(GENERATED_H)
> +$(C_DEP_OBJ): %.dep : %.c command-list.h
> +	$(QUIET_DEP)$(CC) -MF $*.dep -MQ $*.o -MM $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<

I don't think this last dependency is quite right. The .dep for foo.c
must rely not only on foo.c, but also all of its recursive headers (if
it includes foo.h and then foo.h changes to include bar.h, we need to
update the dep file). So it really needs to depend on everything in its
own .dep file, too.

> I.e. here every *.c file depends on a corresponding *.dep file, we
> generate the *.dep files and *.c files seperately, instead of making the
> *.dep a "while we're at it" as we make the *.o.
> [...]
> It also makes "git clean -dxf; make -j8 all" take ~17s on my box instead
> of ~13s.

I definitely think that's easier to reason about, but I'm not surprised
you found that it was slower. To get a more apples-to-apples comparison,
I compiled with -j1. I also use -O0 as my default, which makes the
"real" compilation a relatively smaller portion. I got 47s without your
patch, and 60s with. That seems non-trivial.

I also noticed that "make clean" built all the deps. I think that could
be fixed, but we're piling fixes upon fixes.

So I'm not really sure what we're gaining by splitting this all up.
Given that our original problem is just fixing the test for "do we
support computed dependencies", I think we can solve that pretty easily
and move on.

> So maybe I've talked myself out of there being an inherent dependency
> graph violation with the current schema, i.e. the current one seems like
> an easily solved bug.

Yeah, AFAIK the current system is pretty robust and battle-tested. I'd
be cautious of messing with it unless there's a big gain to be seen.

> One good thing the above approach still gives you is that you can use
> GCC or Clang to make the dependency graph, but then use another compiler
> to compile, i.e. we could have a $(GCC_LIKE) in addition to $(CC), so if
> I'm compiling with xlc or suncc I can still use the present GCC just to
> create the dependency graph.

Keep in mind the worst case in not using the dependency graph is just
that a header update in an incremental compile may cause some extra
compilation. It's really not _that_ bad, and especially so for folks who
aren't actively doing development.

-Peff