[ruby-core:88800] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid

[Eric Wong <normalperson@yhbt.net>]

> https://bugs.ruby-lang.org/issues/13618

Greg, do you think you can try this change for portability?
No rush, though, I will be mostly AFK and Queue/SizedQueue
changes will be trivial with the prep work so far...


Future changes to this feature shouldn't impact, portability.
The main one is [PATCH 2/2]

* [PATCH 1/2] unify sync_waiter, waitpid_state, waiting_fd w/ rb_sched_waiter
   https://80x24.org/spew/20180901131012.22138-1-e@80x24.org/raw

* [PATCH 2/2] Thread::Coro: green threads implemented using fibers
   https://80x24.org/spew/20180901131012.22138-2-e@80x24.org/raw

The following changes since commit 929e9713bbfd76140bced29c6f398904ae9d4a85:

  complex.c: simplify division result (2018-09-01 07:34:31 +0000)

are available in the Git repository at:

  https://80x24.org/ruby.git coro-r64610

for you to fetch changes up to 13c51c9c9ae39dae08497cfe0eb119244c4d2224:

  Thread::Coro: green threads implemented using fibers (2018-09-01 12:56:08 +0000)

    Newest updates in this version:
    
    - new name: Thread::Coro (reject Thriber/Threadlet)
    
    - Fiber#transfer used for switching
      Thanks to funny.falcon for suggestion [ruby-core:87776],
      it made the code better, even :)
    
    - improve IO.select mapping in kqueue/epoll (st_table => ccan/list),
      since there is no need for lookup, only scan
    
    - sync to use MJIT-friendly rb_waitpid (added th->nogvl_runq
      since waitpid(2) can run without GVL)
    
    - "ensure" support.  Only for Thread::Coro, Fiber has no ensure
      (see [Bug #595]).  "ensure" is REQUIRED for auto-scheduling
      safety.  I am not sure if regular Fiber can support "ensure"
      in a way which is both performant and backwards-compatible.
      With Thread::Coro being a new feature, there is no backwards
      compatibility to worry about, so the "ensure" support adds
      practically no overhead
    
    - more code sharing between iom_{select,kqueue,epoll.h}
    
    - switch to rb_hrtime_t for timeouts
    
    - extract timeout API, so non-timeout-arg users can benefit from
      reduced.  This will make Timeout-in-VM support easier and more
      orthogonal to this one.
    
    Changes to existing data structures:
    
    * rb_thread_t
        .runq         - list of Thread::Coros to auto-resume
        .nogvl_runq   - same as above for out-of-GVL use
    
        .waiting_coro - list of blocked Coros, for "ensure"
                        support.  For auto-scheduling, we
                        must have ensure support because
                        Fiber does not yet support ensure
                        <https://bugs.ruby-lang.org/issues/595
    
    * rb_execution_context_t
        .enode        - link to rb_thread_t waiting_coro
    
    rb_vm_t.iom          - Ruby I/O Manager (rb_iom_t) :)
    
    As usual, understanding the data structures first should help
    you understand the code.

----------------------------------------------------------------
Eric Wong (2):
      unify sync_waiter, waitpid_state, waiting_fd w/ rb_sched_waiter
      Thread::Coro: green threads implemented using fibers

> Autoload has similar thread waiting, but I'm not sure if should
> be affected by auto-fiber switching.

Along those lines, I'm not sure what the semantic should be with
regards to Mutex.  Mutex is to protect data from parallel
modifications, but Thread::Coro do not run in parallel.
if a native Thread and Thread::Coro contend on the same Mutex;
what should happen?