[ruby-core:81044] Re: [ruby-cvs:65407] normal:r58236 (trunk): thread.c: comments on M:N threading [ci skip]

[Eric Wong <normalperson@yhbt.net>]

SASADA Koichi <ko1@atdot.net> wrote:
> On 2017/05/08 15:36, Eric Wong wrote:
> > Maybe; if we can avoid GVL and introduce more parallelism.
> >
> > However, I think having one epoll/kqueue FD is better for a
> > whole process; maybe one epoll/kqueue per-core (not per-thread)
> > at maximum.
> >
> > I can easily imagine Ruby doing 100 native threads in one process
> > (8 cores, 10-20 rotational disks, 2 SSD), but 20000-30000 fibers.
> 
> could you elaborate more? 100 epoll threads are not effective?
> Honestly, I have no experience to use epoll/kqueue.

100 epoll FDs is a waste of FDs; especially since it is common
to have a 1024 FD limit.  I already feel bad about timer thread
taking up two FDs; but maybe epoll/kevent can cut reduce that.

In the kernel, every "struct eventpoll" + "struct file" in
Linux is at least 400 bytes of unswappable kernel memory.

Anyways, I've contributed bugfixes to both epoll in the Linux
kernel and also to libkqueue (userspace emulation lib);
and use them both in several projects in and outside of Ruby.

> # context switching to another topic
> 
> > Side note: First, I would like to make fibers smaller.
> > Right now rb_fiber_t stores all of the rb_thread_t
> > struct, but not all fields get used.  I started to work on
> > splitting out to a new struct rb_thread_context_t earlier:
> ...
> > The end goal is to avoid storing all of rb_thread_t inside
> > rb_context_t/rb_fiber_t; and only store rb_thread_context_t.
> > That should reduce memory overhead and maybe make switching
> > faster.
> 
> This is what my goal of Ruby 2.5 (2017) I proposed to my company. If you
> do it, it's great (and I achieved one of my job :)).
> 
> My plan is almost similar but I want to introduce something like
> `mrb_state` which passed to all mruby functions as first argument.
> 
> Do you want to commit this patch before your final goal (lightweight
> fiber switching)?
> 
> 
> FYI: my plan.
> 
> (1) Make separate execution context and make fiber switching lightweight
>   * before 2.5
>   * You named `rb_thread_context_t`, but I don't want to name `thread`
>     so I planned to name it `execution_context` and so on (a bit longer)

OK, I can rename my work-in-progress patch with
s/rb_thread_context_t/rb_execution_context_t/ and commit
later tonight.

> (2-1: extend Fiber) Add Fiber scheduler like you are thinking.
> (2-2: toward Guild) Add `execution_context` as first argument to
>     all C APIs
>   * to keep compatibility, we need to introduce new prefix `rbX_...`
>     for new APIs which receive first argument.
>   * On mruby, `mrb_state` is passed to all of APIs. We need to consider
>     the passed information carefully.

OK, that sounds good.

Also, can you take a look at implementing [Feature #13434]
"better method definition in C API" for rbX_*?
You are more knowledgeable in VM + method definition area;
while I can work on the epoll/kqueue parts for I/O scheduling.

(but I will need to schedule my own human time to work on Ruby :)