[ruby-core:99391] [Ruby master Feature#17059] epoll as IO.select

[ruby-core:99391] [Ruby master Feature#17059] epoll as IO.select

[dsh0416]

Issue #17059 has been reported by dsh0416 (Delton Ding).

----------------------------------------
Feature #17059: epoll as IO.select
https://bugs.ruby-lang.org/issues/17059

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------




-- 
https://bugs.ruby-lang.org/

[ruby-core:99393] [Ruby master Feature#17059] epoll as IO.select's backend

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).

Subject changed from epoll as IO.select to epoll as IO.select's backend

Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling.

----------------------------------------
Feature #17059: epoll as IO.select's backend
https://bugs.ruby-lang.org/issues/17059#change-86797

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------




-- 
https://bugs.ruby-lang.org/

[ruby-core:99568] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[ko1]

Issue #17059 has been updated by ko1 (Koichi Sasada).


does it improve the performance?
My understanding is the advantage of `epoll` is advanced registration.
However, `select` wrapping interface can not use it.

I have no experience to use `epoll`, so correct me it is wrong.

Thanks,
Koichi

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87034

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99600] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).


It should greatly improve the performance.
Advanced registration is a feature of `epoll`,
but the performance is also an important part for it.
The benchmark from libevent shows the performance of epoll and poll or select,
are on a totally different stage.
So this may be important to performance.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87079

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99601] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[shyouhei]

Issue #17059 has been updated by shyouhei (Shyouhei Urabe).


@ko1 is not talking about efficiency of epoll in general, but questioning that of your patch.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87080

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99602] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).


In general, event handling gems like nio4r could provide a similar `select` interface with multiple backends including select, kqueue and epoll support.
On the side of Ruby meta-programming, this part is easy to be implemented, and could provide a much better performance comparing to the default IO.select.
But since Ruby merged the Fiber scheduler recently, the IO.select injection from the core library may be important to provide extra performance.
From the patch, Ruby does use the POSIX `fdset_t` in some platforms, but Ruby also defines its own structs on some other platforms for non-standard select implementation.
Since the `IO.select` has seperated code with macro to use these customized struct. We may also create another macro branch for the epoll compatible implementation.
This is my current idea, and I'm working on implementing this.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87081

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99603] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).

File epoll.h added

Update the WIP implementation

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87082

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99604] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[samuel]

Issue #17059 has been updated by ioquatix (Samuel Williams).


This is a nice idea, and I've considered exposing `wait_select` on the scheduler interface.

I'd suggest we try to go down this route.

The scheduler interface can cache the I/O registration... so in theory it addresses @ko1's concerns.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87083

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99608] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).


Thanks for advice.

To separate the process of registration and wait is a good idea for performance.

Since even the `select` itself could also take advantages from this,

and simplify the whole I/O multiplexing process.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87087

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99610] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[eregontp]

Issue #17059 has been updated by Eregon (Benoit Daloze).


I'm unclear how using `epoll` can help for the user calling `IO.select`.
With the API of select(), I'd expect `epoll` is no faster than `select()`.

Regarding the Fiber scheduler, then I think new event is needed, calling epoll() instead of select() wouldn't solve anything, right?

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87089

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99611] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[ko1]

Issue #17059 has been updated by ko1 (Koichi Sasada).


I want to know the general idea how to use `epoll` for `IO.select` backend.

```C
#include <stdlib.h>
#include <stdio.h>

#define _GNU_SOURCE
#include <unistd.h>

#include <sys/resource.h>
#include <poll.h>
#define N 2000 // 10k

static void
task_poll(int fds[])
{
    struct pollfd pfd[N];
    for (int i=0; i<N; i++) {
        struct pollfd *p;
        // in
        p = &pfd[i];
        p->fd = fds[i*2];
        p->events = POLLIN;
    }

    int r = poll(&pfd[0], N, 0);
    if (r==0) {
        // timeout
    }
    else if (r>0) {
        for (int i=0; i<N*2; i++) {
            fprintf(stderr, "%d %d (%d)\n", i, pfd[i].fd, (int)pfd[i].revents);
        }
    }
    else {
        fprintf(stderr, "poll (RLIMIT_NOFILE:%d and N:%d)\n", (int)RLIMIT_NOFILE, N*2);
        exit(1);
    }
}

#include <sys/epoll.h>

static void
task_epoll(int fds[])
{
    struct epoll_event events[N];
    int efd = epoll_create(N);
    if (efd < 0) {
        perror("epoll");
        exit(1);
    }

    for (int i=0; i<N; i++) {
        struct epoll_event *e = &events[i];
        e->events = EPOLLIN;
        e->data.fd = fds[i*2];
        if (epoll_ctl(efd, EPOLL_CTL_ADD, fds[i*2], e) < 0) {
            perror("epoll_ctl");
            exit(1);
        }
    }

    int r = epoll_wait(efd, events, N, 0);
    if (r == 0) {
        // timeout
    }
    else if (r > 0) {
        for (int i=0; i<r; i++) {
            fprintf(stderr, "%d fd:%d\n", i, events[i].data.fd);
        }
    }
    else {
        perror("epoll_wait");
        exit(1);
    }

    // clear
    close(efd);
}


int main(void)
{
    int fds[N * 2];
    int i;
    for (i=0; i<N; i++) {
        if (pipe(&fds[i*2]) < 0) {
            perror("pipe");
            fprintf(stderr, "i:%d\n", i);
            exit(1);
        }
    }

    for (i=0; i<1000 * 10; i++) {
        // task_xxx emulates IO.select

        // task_poll(fds);  // real    0m0.537s
        // task_epoll(fds); // real    0m11.191s
    }

    return 0;
}
```

`epoll` version is x20 slower on my machine.
any misunderstanding?

(`efd` can be reusable, but I'm not sure how to clear all registered fds)


----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87090

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99612] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[ko1]

Issue #17059 has been updated by ko1 (Koichi Sasada).


I understand `epoll` will help by introducing new event handling APIs.
But not sure why `IO.select` can improve the performance with `epoll`.
This is my question by first comment and maybe https://bugs.ruby-lang.org/issues/17059#note-13 is same.


----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87091

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99616] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).


The benchmark looks good. I've tested with similar code, and it's 46x slower on my machine.
It looks like `epoll` is highly depended on the time that `epoll_ctl` engaged.

Since the scheduler now have other registration control including `rb_io_wait_readable` and `rb_io_wait_writable` are introduced in the current `Scheduler`.
I would try to use these methods to deal with the registration then, and replace the `IO.select` in the `Scheduler#run` for performance.

Is this a proper way to implement then?

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87095

* Author: dsh0416 (Delton Ding)
* Status: Open
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99618] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[ko1]

Issue #17059 has been updated by ko1 (Koichi Sasada).

Status changed from Open to Rejected

> Since the scheduler now have other registration control including rb_io_wait_readable and rb_io_wait_writable are introduced in the current Scheduler.
> I would try to use these methods to deal with the registration then, and replace the IO.select in the Scheduler#run for performance.

I'm not sure what is your idea, but at least I reject this ticket because `IO.select` is not good place to use `epoll`.

Maybe design Ruby-level event handling API and use it by scheduler is the best.

```ruby
s = IO::Selector
s.add(io1, 'r')
s.add(io2, 'w')
s.add(io3, 'rw')
s.wait #=> ready io array
```

for example.


----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87097

* Author: dsh0416 (Delton Ding)
* Status: Rejected
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99624] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[eregontp]

Issue #17059 has been updated by Eregon (Benoit Daloze).


dsh0416 (Delton Ding) wrote in #note-16:
> I would try to use these methods to deal with the registration then, and replace the `IO.select` in the `Scheduler#run` for performance.

Where do you see `IO.select` in the `Scheduler`?
Here? https://github.com/ruby/ruby/blob/701217572f865375b137d2830d4da0c3e78de046/test/fiber/scheduler.rb#L44
That's a test scheduler, and using `IO.select()` is good enough for prototyping but basically nothing else.

@ioquatix it seems worth clarifying that with a comment there.
Might also be worth linking to the async scheduler as a real-world example there or in `doc/fiber.rdoc`.

Real schedulers like the one for `async` use epoll/kqueue/libev/etc internally, for instance by using `nio4r`.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87105

* Author: dsh0416 (Delton Ding)
* Status: Rejected
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99625] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[eregontp]

Issue #17059 has been updated by Eregon (Benoit Daloze).


In other words I don't think we need to have selectors in Ruby core.
That's part of the beauty of this new scheduler API: it lets you implement it in various way, including how do you want to wait for IO.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87106

* Author: dsh0416 (Delton Ding)
* Status: Rejected
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:99628] [Ruby master Feature#17059] epoll as the backend of IO.select on Linux

[dsh0416]

Issue #17059 has been updated by dsh0416 (Delton Ding).


Yes. I was just figured out that the scheduler is an example in the tests, where the real scheduler is designed to be separated from the ruby-core.

----------------------------------------
Feature #17059: epoll as the backend of IO.select on Linux
https://bugs.ruby-lang.org/issues/17059#change-87109

* Author: dsh0416 (Delton Ding)
* Status: Rejected
* Priority: Normal
----------------------------------------
Current Ruby's `IO.select` method calls POSIX `select` API directly. With the new non-blocking scheduler, this may be the bottleneck of the I/O scheduling. For keeping the backward compatibilty of the current `IO.select` methods, a proposal may be to create a "duck" `select` which uses the `epoll_wait` as the backend.

One tricky part is that the `fd_set` described in POSIX is write-only, which means it is impossible to iterate for generating the `epoll_event` argument for `epoll_wait`. But similar to the large-size select situation, we could define our own `rb_fdset_t` struct in this case, and implement the following APIs.

```
void rb_fd_init(rb_fdset_t *);
void rb_fd_term(rb_fdset_t *);
void rb_fd_zero(rb_fdset_t *);
void rb_fd_set(int, rb_fdset_t *);
void rb_fd_clr(int, rb_fdset_t *);
int rb_fd_isset(int, const rb_fdset_t *);
void rb_fd_copy(rb_fdset_t *, const fd_set *, int);
void rb_fd_dup(rb_fdset_t *dst, const rb_fdset_t *src);
int rb_fd_select(int, rb_fdset_t *, rb_fdset_t *, rb_fdset_t *, struct timeval *);
```

TODO:
1. Implement the fd_set with dynamic allocated fds.
2. Implement the epoll with select API.
3. Edit io.c to use the customized fd_set struct.

I'm trying to work on a branch for this. Any suggestions for this?

---Files--------------------------------
epoll.h (3.62 KB)
epoll.h (6.44 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:100072] Re: [Ruby master Feature#17059] epoll as IO.select

[Eric Wong]

dsh0416@gmail.com wrote:
> Feature #17059: epoll as IO.select
> https://bugs.ruby-lang.org/issues/17059

Try ppoll(2) to replace IO.select, instead.  It also works on
newer FreeBSD.

To use epoll(7) effectively, you really need a scheduler API and
probably restructure programs.  Furthermore, the Level-Triggered
(default) epoll behavior seems to be cause for many complaints
and FUD about epoll in general.

I recommend sticking to EPOLLONESHOT mostly, and EPOLLET for
unidirectional cases.

  * EPOLLONESHOT - easiest for bidirectional sockets,
    hard to get wrong once you use it

  * EPOLLET - Use it for listen sockets, pipes and FIFOs.
    That is, one-way stuff that's always EPOLLIN or EPOLLOUT-only,
    but can never be BOTH EPOLLIN or EPOLLOUT.  Theoretically it
    could be faster than EPOLLONESHOT for bidirectional sockets
    due to fewer syscalls, but tricky-to-use as it depends on client
    traffic patterns (which clients can abuse).

Fwiw: I've read and understood fs/eventpoll.c in the Linux source;
it's fairly self-contained and should be easy-to-understand to
anybody remotely familiar with C.  There's a lot hate on epoll out there
and I strongly suspect the default (level-trigger) behavior is to blame.