Re: Module suggestion: timeout

[Bruno Haible <bruno@clisp.org>]

Hi Marc,

Marc Nieper-Wißkirchen wrote:
> Gnulib contains a module named valgrind

More precisely, it's called 'valgrind-tests'. See
https://www.gnu.org/software/gnulib/manual/html_node/Using-valgrind-automatically.html

> that allows the easy use of
> Valgrind for tests whenever it is supported by the build system.
> 
> I would suggest to add a similar module named timeout that sets the
> variable TIMEOUT with suitable defaults whenever the GNU coreutils timeout
> program (or an equivalent) is available.
> 
> This would allow test runners to be unconditionally prefixed with
> $(TIMEOUT) so that on supporting systems, tests are killed (and an error is
> reported) whenever they run for too long (mostly because of a logic error
> causing an infinite loop).

The situations are a bit different:

1) For the timeout, there is a coding idiom that needs little code and works
also when the 'timeout' program is not present:

#if HAVE_DECL_ALARM
  /* Declare failure if test takes too long, by using default abort
     caused by SIGALRM.  */
  int alarm_value = 600;
  signal (SIGALRM, SIG_DFL);
  alarm (alarm_value);
#endif

2) For the timeout, different tests need different timeouts. Whereas valgrind
typically does not need per-program arguments.

3) If a way to enable a timeout exists, it is typically never useful to run a
unit test without a timeout. Whereas if valgrind is available, it is still
useful to run a unit test without valgrind:
  - the unit test runs much faster without valgrind,
  - the unit test might not be prepared to cope with the extra '==...' lines
    output to stderr.

So, I don't think the "let's treat timeout like valgrind" approach is going
to work. Instead, you need to design a way to deal with timeouts, independently.

Bruno