[ruby-core:66366] Re: [ruby-trunk - Bug #10460] Segfault instead of stack level too deep

[Eric Wong <normalperson@yhbt.net>]

arne@arnebrasseur.net wrote:
> Mutant will often generate "broken" code, that's how it works, so
> endless recursion could be the result. It needs to be able to detect
> somehow that things go wrong. A segfault is actually not the biggest
> problem. Mutant forks off workers, so if the worker dies it can assume
> something went wrong. Sometimes the code in this ticket also causes
> Ruby to get stuck in a futex. In that case the worker is "stuck" and
> Mutant becomes unusable.

The freezing is likely caused by the stack overflow corrupting memory
used by a mutex, putting the mutex in an unrecoverable state.
You're better off to detecting lockups in the parent process via
periodic checks.

> > We may increase the size of the guard area; but that costs memory.
> > Right now, on (most) Linux systems, this guard costs 4K (one page)
> > per-thread.
> 
> Could you tell me where in the code I can see this? I would love to
> investigate this more. Thanks!

On GNU/Linux systems, this is done by glibc upon thread creation.

We only set the stack size via pthread_attr_setstacksize in
thread_pthread.c, but we could also call pthread_attr_setguardsize in
the same place.