[ruby-core:81244] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip]

[Eric Wong <normalperson@yhbt.net>]

Work-in-progress patch:

   https://80x24.org/spew/20170519042738.7174-1-e@80x24.org/raw

Currently, IO scheduling seems to work, waitpid/sleep/other scheduling
is not done, yet; but we do not need to support everything at
once during dev.

main API:

    Fiber#start -> enable auto-scheduling and run Fiber until it
                   automatically yields (due to EAGAIN/EWOULDBLOCK)

The following behave like their Thread counterparts:

    Fiber#join - run internal scheduler until Fiber is terminated
    Fiber#value - ditto

    Fiber#run (in prelude.rb)
    Fiber.start (ditto)

I think we can iron out the internal APIs and behavior, first,
and gradually add support for auto-Fiber.yield points.

Right now, it takes over rb_wait_for_single_fd() function
if the running Fiber is auto-enabled (cont.c::rb_fiber_auto_sched_p)

Changes to existing functions are minimal.

New files (important structs and relations should be documented):

    iom.h - internal API for the rest of RubyVM (incomplete?)
    iom_common.h - common stuff internal to iom_*.h
    iom_select.h - select()-specific pieces

Changes to existing data structures:

    rb_thread_t.afhead   - list of fibers to auto-resume
    rb_fiber_t.afnode    - link to th->afhead
    rb_vm_t.iom          - Ruby I/O Manager (rb_iom_t) :)

Besides rb_iom_t, all the new structs are stack-only and
relies extensively on ccan/list for O(1) insert/delete.

Right now, I reuse some static functions in thread.c,
so thread.c includes iom_select.h

TODO:

    Hijack other blocking functions (waitpid, IO.select, ...)
    iom_epoll.h + iom_kqueue.h (easy once iom.h definitions are done)

I am using "double" for timeout since it is more convenient for
arithmetic like parts of thread.c.   Most platforms have good FP,
I think.  Also, all "blocking" functions (rb_iom_wait*) will
have timeout support

git repo info:

The following changes since commit c26a9a733848a0696976bb98abfe623e15ba2979:

  Fix strange indentation (2017-05-18 15:13:30 +0000)

are available in the git repository at:

  git://80x24.org/ruby.git iom_select

for you to fetch changes up to 8ee92fbc908fe67f52372443a1492fc490431de0:

  auto fiber scheduling and friends (VERY LIGHTLY TESTED) (2017-05-19 03:51:28 +0000)

----------------------------------------------------------------
Eric Wong (1):
      auto fiber scheduling and friends (VERY LIGHTLY TESTED)

 common.mk         |   3 +
 configure.in      |   2 +-
 cont.c            | 156 +++++++++++++++++++-
 include/ruby/io.h |   2 +
 iom.h             |  82 +++++++++++
 iom_common.h      |  93 ++++++++++++
 iom_select.h      | 419 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 prelude.rb        |  12 ++
 thread.c          |  22 +++
 vm.c              |   7 +-
 vm_core.h         |   5 +-
 11 files changed, 795 insertions(+), 8 deletions(-)
 create mode 100644 iom.h
 create mode 100644 iom_common.h
 create mode 100644 iom_select.h
 (I will revert the -O0 change in configure.in, of course :)