ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
From: dsh0416@gmail.com
To: ruby-core@ruby-lang.org
Subject: [ruby-core:99602] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux
Date: Sun, 16 Aug 2020 10:24:09 +0000 (UTC)	[thread overview]
Message-ID: <redmine.journal-87081.20200816102409.13868@ruby-lang.org> (raw)
In-Reply-To: redmine.issue-17059.20200729153305.13868@ruby-lang.org

Issue #17059 has been updated by dsh0416 (Delton Ding).


In general, event handling gems like nio4r could provide a similar `select` interface with multiple backends including select, kqueue and epoll support.
On the side of Ruby meta-programming, this part is easy to be implemented, and could provide a much better performance comparing to the default IO.select.
But since Ruby merged the Fiber scheduler recently, the IO.select injection from the core library may be important to provide extra performance.
From the patch, Ruby does use the POSIX `fdset_t` in some platforms, but Ruby also defines its own structs on some other platforms for non-standard select implementation.
Since the `IO.select` has seperated code with macro to use these customized struct. We may also create another macro branch for the epoll compatible implementation.
This is my current idea, and I'm working on implementing this.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87081

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)


-- 
https://bugs.ruby-lang.org/

  parent reply	other threads:[~2020-08-16 10:24 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-29 15:33 [ruby-core:99391] [Ruby master Feature#17059] epoll as IO.select dsh0416
2020-07-29 15:34 ` [ruby-core:99393] [Ruby master Feature#17059] epoll as IO.select's backend dsh0416
2020-08-12  7:38 ` [ruby-core:99568] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux ko1
2020-08-16  1:12 ` [ruby-core:99600] " dsh0416
2020-08-16  9:02 ` [ruby-core:99601] " shyouhei
2020-08-16 10:24 ` dsh0416 [this message]
2020-08-16 21:08 ` [ruby-core:99603] " dsh0416
2020-08-16 23:49 ` [ruby-core:99604] " samuel
2020-08-17  7:37 ` [ruby-core:99608] " dsh0416
2020-08-17  9:41 ` [ruby-core:99610] " eregontp
2020-08-17  9:49 ` [ruby-core:99611] " ko1
2020-08-17  9:53 ` [ruby-core:99612] " ko1
2020-08-17 12:00 ` [ruby-core:99616] " dsh0416
2020-08-17 17:37 ` [ruby-core:99618] " ko1
2020-08-18  9:07 ` [ruby-core:99624] " eregontp
2020-08-18  9:09 ` [ruby-core:99625] " eregontp
2020-08-18 16:05 ` [ruby-core:99628] " dsh0416
2020-09-22 17:43 ` [ruby-core:100072] Re: [Ruby master Feature#17059] epoll as IO.select Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.ruby-lang.org/en/community/mailing-lists/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=redmine.journal-87081.20200816102409.13868@ruby-lang.org \
    --to=ruby-core@ruby-lang.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).