Re: [PATCH] Makefile: add and use the ".DELETE_ON_ERROR" flag

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Thu, Jun 24 2021, Jeff King wrote:

> On Thu, Jun 24, 2021 at 03:53:51PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> >> ..and "this behavior is really annoying on one platform we target, and
>> >> the fix is rather trivial".
>> >
>> > Yeah, that's a fine reason, too. I'm not entirely clear on what the
>> > problem is, though, or why this is the best solution (I expect you
>> > probably explained it in an earlier thread/series, but if so it went in
>> > one ear and out the other on my end).
>> 
>> On *nix systems you can open a file, and unlink() it in another process,
>> on Windows this is an error.
>> 
>> AIX has its own variant of this annoying behavior, you can't clobber (or
>> open for writing) binaries that are currently being run, you can unlink
>> and rename them though.
>
> Ah, right. Thanks for refreshing me.
>
> TBH, I don't find this that serious a problem. Your compile will fail.
> But is rebuilding in the middle of a test run something it's important
> to support seamlessly? It seems like a bad practice in the first place.

Yeah I think so, and I think it's good practice, it enabled development
workflows that you and Junio clearly don't use (which is fine), but
which are useful to others. For me it would result in more total
testing, and earlier catching of bugs, not less.

Quoting an earlier mail of yours[1]:

    I think having the test suite loudly complain is a good way to
    remind you that you have not in fact run the whole suite on a given
    build.

It's useful as you're programming you save/compile, and have the tests
run in a loop in the background, and are alerted if they start
failing.

That's not really possible with git currently without having that loop
continually push to another workdir, making it work in one checkout
helps for some workflows.

Yes it could allow you to run format-patch and send-email while you're
50% through a full loop, or not just run the full tests then, but at
some point I think we've got to assume some basic competency from
people. We also have CI running the full tests, and I could have just
run tests 0000-5000, compiled, and then run 5001-9999.

As a mechanism to prevent this it's not even reliable, it won't always
prevent this due to races, you'd need to e.g. issue a "git version" at
the start of a run, then "Bail Out!" if you detect it being different at
the end.

> It would likewise be a problem if you were running regular git commands
> straight out of your build directory. And we do support that, but IMHO
> it is not that important a use case.
>
> So again, I'm not all that opposed to atomic rename-into-place
> generation. But the use case doesn't seem important to me.

I guess because we use computers differently. I often have say a full
test run in one window, see a failure scroll by, re-make in another
window, test in a third, it's annoying to have to go back & forth and
stop/start things. I typically run the "main" one as a while-loop.

>> So without that "mv $@ $@+" trick you can't recompile if you have a
>> concurrent test (that you don't care about failing) running, and we have
>> bugs in our tests where e.g. "git-daemon" gets lost and won't get killed
>> on that platform, so you can't recompile and test without tracking it
>> down and killing it.
>
> The "git-daemon" thing sounds like a separate bug that is maybe
> exacerbating things. But we'd want to fix it anyway, since even without
> blocking compilation, it will cause a re-run of the tests to use the
> wrong version (and either fail, or hit the auto-skip behavior). I've run
> into this with apache hanging around after tests were killed (we do try
> to clean up, but depending how the script is killed, that may or may not
> succeed).

Yes, it's a bug that should be fixed. Right now if I try to login and
fix it (and numerous other bugs) my second full test run is guaranteed
to be impeded by having to track down and kill things.

The in-place-move works around that perfectly, so as a chicken-and-egg
problem of getting to a point where it's not so annoying to fix things
that I give up I suggested this rather trivial "&& mv $@+ $@" change was
something worth carrying.

1. https://lore.kernel.org/git/YEZsON5OxUiDkqPG@coredump.intra.peff.net/