Re: [PATCH v2 1/5] Makefile: rename objects in-place, don't clobber

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Mon, Mar 29 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Per the log of changes to the Makfile and Junio's recent comment about
>> [1] why that pattern got introduced it was for a different reason
>> entirely, i.e. ("[]" edits are mine, for brevity):
>>
>>     [T]hat age old convention [...] is spelled [as]:
>>
>>     	thing:
>>     		rm -f thing thing+
>>     		prepare contents for thing >thing+
>
> Did I say that?  I recall I specifically avoided the "redirection"
> because this is *NOT* shell-script only principle.  If a command is
> so broken that "cmd -o thing" that fails to work correctly leaves a
> broken output in thing, then the pattern should be applied and made
> to "cmd -o thing+ && mv thing+ thing".

[I see this was already noted downthread...]

> On the other hand, if "cmd" refrains from leaving a half-baked
> result in "-o thing" (and I believe $(CC) is well-behaved even on
> AIX), I do not think it is a good idea to use that pattern.  One
> version of AIX may refuse to overwrite a file in use because
> creat("thing") that is necessary for "-o thing" may fail while
> "thing" is in use), but another system may refuse to rename over a
> file in use (i.e. "-o thing+" into a brand new "thing+" may be OK
> but the final "mv thing+ thing" may fail).  So pretending that it is
> a cure for broken use case is cluttering Makefile for no real
> benefit and leading readers into confused thinking.

It does fix this issue entirely on AIX. So it's a cure for the broken
case.

I can assure you that not having to regularly log in to the GCC farm AIX
box and remember how to deal with IBM's ps/kill etc. just to do another
build/test is a real benefit :)

Whereas maybe there's another system we're not fixing with this patch,
but does it matter? I don't see how it would make things worse for that
OS, if it exists. But it sounds like it's just a hypothetical.

>>     		mv thing+ thing
>>
>>     It protects us from a failure mode where "prepare contents for
>>     thing" step is broken and leaves a "thing" that does not work, but
>>     confuses make that make does not need to rebuild it, if you wrote it
>>     as such:
>>
>>     	thing:
>>     		prepare contents for thing >thing
>>
>>     [It might leave behind a corrupt 'thing'.] In any case, it is not
>>     "we are trying to make thing available while it is being
>>     rewritten" at all.
>>
>> That makes perfect sense for shellscripts, but as this change shows
>> there's other good reasons to use this age old convention that weren't
>> considered at the time.
>
> So, no, the age old convention may have considered and does not
> apply to such a use case.

The reason I mentioned this was to specifically address the implied "why
would we need this?" part of your E-Mail that you're quoting.

I think that started out as us talking past one another, so let me try
to summarize.

I was basically saying "here's a well-known command idiom" that can be
used for XYZ.

I think you're basically saying "in git.git, I introduced this idiom to
deal with problem ABC".

ABC and XYZ aren't incompatible. I'm just saying this can and does solve
both problems[continued below].

>>  git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
>> -	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
>> -		$(filter %.o,$^) $(LIBS)
>> +	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@+ $(ALL_LDFLAGS) \
>> +		$(filter %.o,$^) $(LIBS) && \
>> +	mv $@+ $@
>
> Really, does anybody else use "$(CC) -o $@" in such a way in their
> Makefile?  Having to do this smells simply crazy (I am not saying
> you are crazy---the platform that forces you to write such a thing
> is crazy).

Yes, if you do say a Google search for "Cannot open or remove a file
containing a running program" you'll find that there's 15k results of
people basically (re)discovering this problem in porting their software
to AIX, and the solutions being some variant of "yes AIX sucks, just use
this 'cmd >x+ && mv x+ x' trick".

You can also invoke a "slibclean" program to reticulate AIX's splines,
but I thought this sucked less.