[ruby-core:81042] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip]

[SASADA Koichi <ko1@atdot.net>]

On 2017/05/08 15:36, Eric Wong wrote:
> Maybe; if we can avoid GVL and introduce more parallelism.
>
> However, I think having one epoll/kqueue FD is better for a
> whole process; maybe one epoll/kqueue per-core (not per-thread)
> at maximum.
>
> I can easily imagine Ruby doing 100 native threads in one process
> (8 cores, 10-20 rotational disks, 2 SSD), but 20000-30000 fibers.

could you elaborate more? 100 epoll threads are not effective?
Honestly, I have no experience to use epoll/kqueue.


# context switching to another topic

> Side note: First, I would like to make fibers smaller.
> Right now rb_fiber_t stores all of the rb_thread_t
> struct, but not all fields get used.  I started to work on
> splitting out to a new struct rb_thread_context_t earlier:
...
> The end goal is to avoid storing all of rb_thread_t inside
> rb_context_t/rb_fiber_t; and only store rb_thread_context_t.
> That should reduce memory overhead and maybe make switching
> faster.

This is what my goal of Ruby 2.5 (2017) I proposed to my company. If you
do it, it's great (and I achieved one of my job :)).

My plan is almost similar but I want to introduce something like
`mrb_state` which passed to all mruby functions as first argument.

Do you want to commit this patch before your final goal (lightweight
fiber switching)?


FYI: my plan.

(1) Make separate execution context and make fiber switching lightweight
  * before 2.5
  * You named `rb_thread_context_t`, but I don't want to name `thread`
    so I planned to name it `execution_context` and so on (a bit longer)
(2-1: extend Fiber) Add Fiber scheduler like you are thinking.
(2-2: toward Guild) Add `execution_context` as first argument to
    all C APIs
  * to keep compatibility, we need to introduce new prefix `rbX_...`
    for new APIs which receive first argument.
  * On mruby, `mrb_state` is passed to all of APIs. We need to consider
    the passed information carefully.

Thanks,
Koichi

-- 
// SASADA Koichi at atdot dot net