Re: [PATCH] nptl: Fix deadlock on atfork handler which calls dlclose (BZ#24595)

[Florian Weimer <fweimer@redhat.com>]

* Adhemerval Zanella:

> We can go back to old behaviour of using a lock-free registration on the
> atfork lists, make a lock-free copy on fork, and handle the synchronization
> with __unregister_atfork using locks plus futexes.  It would require all the
> old complexity of mixing lock-free algorithms with locks accesses plus using 
> an unbounded alloca on fork (maybe we could just use malloc to allocate
> the backup list, my understanding it is avoid to to add another issue to
> make fork async-signal-safe).

I think we can avoid the async-signal-safety issue (for now) by not
using locks in single-threaded mode.  pthread_atfork is not required to
be async-signal-safe, so we need not worry about fork/pthread_at_fork
interactions in single-threaded processes, beyond the modification of
the handler list from the handler itself.

Maybe we can avoid making the copy for every fork call, using some sort
of copy-on-write list?  On the reader side, the protocol would look like
this:

  lock the fork handler mutex
    get pointer to the dynlist head from a global variable
    increment the reference counter next to the dynlist head
  unlock the fork handler mutex

  perform all the fork work, traversing the list as needed
  (without any locking)

  lock the fork handler mutex
    decrement the reference counter
    if the counter is zero, delocate the entire data structure
      (list and struct containing list head plus reference counter)
  unlock the fork handler mutex

On the registration/unregistration side (pthread_atfork,
__unregister_atfork), we would do this:

  lock the fork handler mutex
    get pointer to the dynlist head from the global variable
    if the reference counter is 1:
      modify the list in place
    else:
      make a modified copy of the list
      update the global variable to point to it
      decrement the reference counter in the original list
  unlock the fork handler mutex

(The reference counter in the quiet state would have to be 1, because
the list is referenced from the global variable.)

I think this is still simpler than the original scheme.  It also ensures
that we are not modifying the list during the traversal in fork.
Instead, we update a list that will be used for future forks.

What do you think?

Thanks,
Florian