Re: [PATCH v2 5/5] cmake: increase time-out for a long-running test

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Tue, Aug 23 2022, Johannes Schindelin via GitGitGadget wrote:

> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> As suggested in
> https://github.com/git-for-windows/git/issues/3966#issuecomment-1221264238,
> t7112 can run for well over one hour, which seems to be the default
> maximum run time at least when running CTest-based tests in Visual
> Studio.
>
> Let's increase the time-out as a stop gap to unblock developers wishing
> to run Git's test suite in Visual Studio.
>
> Note: The actual run time is highly dependent on the circumstances. For
> example, in Git's CI runs, the Windows-based tests typically take a bit
> over 5 minutes to run. CI runs have the added benefit that Windows
> Defender (the common anti-malware scanner on Windows) is turned off,
> something many developers are not at liberty to do on their work
> stations. When Defender is turned on, even on this developer's high-end
> Ryzen system, t7112 takes over 15 minutes to run.
>
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>  contrib/buildsystems/CMakeLists.txt | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
> index 29d7e236ae1..b1306f95256 100644
> --- a/contrib/buildsystems/CMakeLists.txt
> +++ b/contrib/buildsystems/CMakeLists.txt
> @@ -1088,4 +1088,8 @@ foreach(tsh ${test_scipts})
>  		WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}/t)
>  endforeach()
>  
> +# This test script takes an extremely long time and is known to time out even
> +# on fast machines because it requires in excess of one hour to run
> +set_tests_properties("${CMAKE_SOURCE_DIR}/t/t7112-reset-submodule.sh" PROPERTIES TIMEOUT 4000)
> +
>  endif()#BUILD_TESTING

I don't see per [1] that it would have any negative effect to just bump
the timeout a lot more, and for all the tests.

If we're running into 3600 seconds, then setting this to 4000 seconds
seems to be a smal stopgap at best. That's just over a 10% increase, so
if one person ran into it it 3600 someone with a slightly slower system
should be running into the same still, or if we just add a few more
tests to 7112 (or some other slow test).

So why not set this ta 3600*10 or whatever for *all* of the test scripts
instead of playing whack-a-mole?

1. https://cmake.org/cmake/help/latest/prop_test/TIMEOUT.html