[ruby-core:71666] [Ruby trunk - Bug #11549] Object allocation during garbage collection phase terminates the Ruby process

[charlez.leu@gmail.com]

Issue #11549 has been updated by Charles Leu.


Questions/Clarifications please:
* Are you saying there exists a single specific thread, and only that specific single thread will run garbage collection?
* Are you saying that if any C extension invokes rb_thread_call_without_gvl(), and that a different thread performs I/O and awaits I/O completion (that permits a yet different thread to run), then there exists the possibility of an interpreter crash (due to the different thread that was switched to allocating an object)?
* Please provide/add clarifications, thanks.

Notes:
* This problem is readily reproducible on multiple versions of Ruby 1.9.3 through 2.2.x.
* When I have time to pare down the apps that yield the described problem, I'll make it/them available (ETA currently unknown).
* My last question was specific to writing multi-threaded apps, and was looking for:
> * Guidelines for writing multi-threaded Ruby apps.
> * Information regarding how the Ruby interpreter handles context switches.
> * Special considerations for process SIGNAL handling, exception handling, recovery and termination conditions.
* While multiple processes are a means of implementing parallelism, they are heavy weight, and often consume far more system resource than desired.

Thank You for your support.

----------------------------------------
Bug #11549: Object allocation during garbage collection phase terminates the Ruby process
https://bugs.ruby-lang.org/issues/11549#change-55068

* Author: Charles Leu
* Status: Feedback
* Priority: Normal
* Assignee: Koichi Sasada
* ruby -v: ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
Multi-Threaded Ruby apps are often problematic; especially so when utilizing thread pools, and scheduling work to worker threads.

RE: attached file ruby_2.2.3_obj_alloc_gc_bug.txt

Following is the section of sap_consumer_control.rb that is presented by the Ruby interpreter as being the current execution context when the problem occurs.

408: loop do
409:   @worker_threads.schedule(@work_queue.pop, &@consumer)
410:   @sap_packets_consumed += 1
411: end

Notes:
* @work_queue is a Ruby Queue (allocated within the main thread) into which a producer thread places work requests.
* @worker_threads is a thread pool (allocated within the main thread).
* @worker_threads schedule method simply puts a work request into the thread pool's internal work queue.  One of the worker threads within the thread pool will consume/effect the work request, by executing the specified consumer Proc.
* The main program thread simply loops forever scheduling work to thread pool threads.
* It appears that an object is being allocated by virtue of the @work_queue.pop

Questions:
* Does Ruby garbage collection potentially run within each thread context?
* If answer to prior question is 'Yes', then how can object allocation be prevented when a sibling thread is attempting garbage collection?
* How does one write multi-threaded Ruby apps on multi-core systems that permit multiple-concurrent execution contexts that don't employ a mutex to effectively single thread the entire app? 

---Files--------------------------------
ruby_2.2.3_obj_alloc_gc_bug.txt (49.1 KB)


-- 
https://bugs.ruby-lang.org/