[ruby-core:90624] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[ruby-core:90624] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been reported by sylvain.joyeux (Sylvain Joyeux).

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?



-- 
https://bugs.ruby-lang.org/

[ruby-core:90642] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).

File 0001-dynamically-modify-the-timer-thread-period-to-accoun.patch added

This is the implementation of option (4) on 2.5.3 (since that's what I am using). If it has a chance to be accepted, I will forward-port it to 2.6.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-75808

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91087] [Ruby trunk Bug#15438] Threads can't switch faster thanTIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


Anyone interested on the subject ?

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-76315

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91090] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[nobu]

Issue #15438 has been updated by nobu (Nobuyoshi Nakada).


Your patch can't apply to the latest code.
Could you rebase it?

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-76321

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91102] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).

File 0001-2.6-fix-handling-of-negative-priorities.patch added

Hi nobu. Thanks for looking into this !

Here's the patch for 2.6. I would be glad if you could consider also applying the other patch to 2.5.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-76337

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91131] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


Quick question ... what is the preferred method to provide patches now ? I'm seeing that there are PR on GitHub ... Should I propose the patch here instead ?

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-76362

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91135] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[nobu]

Issue #15438 has been updated by nobu (Nobuyoshi Nakada).


sylvain.joyeux (Sylvain Joyeux) wrote:
> Here's the patch for 2.6. I would be glad if you could consider also applying the other patch to 2.5.

Thank you so much.

In thread_win32.c:

```diff
-#define TIME_QUANTUM_USEC (10 * 1000)
+#define TIME_QUANTUM_USEC_BASE (100 * 1000)
```
Why 10 times?

```diff
+#define TIME_QUANTUM_USEC TIME_QUANTUM_USEC
```
`_BASE` is missing at the last?

```diff
+static const rb_hrtime_t TIME_QUANTUM_NSEC = &TIME_QUANTUM_USEC * 1000;
```
The address of a lvalue is invalid, and the multiplication too.
Just `&` isn't needed?

sylvain.joyeux (Sylvain Joyeux) wrote:
> Quick question ... what is the preferred method to provide patches now ? I'm seeing that there are PR on GitHub ... Should I propose the patch here instead ?

Anywhere you like.



----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-76366

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91787] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


Follow-up PR: https://github.com/ruby/ruby/pull/2087

I apologize for Windows not building, I don't have a windows machine to try it on. Unfortunately, the Azure Pipeline build on GH failed to fetch the repository, and there doesn't seem to be a way to restart it.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77062

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91788] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[takashikkbn]

Issue #15438 has been updated by k0kubun (Takashi Kokubun).


> I apologize for Windows not building

Both Linux and macOS on Travis are red. It's fine that you ignore our unstable Azure Pipeline for now, but you can still use AppVeyor for checking Windows. Please just try to make Travis and AppVeyor green.

> Unfortunately, the Azure Pipeline build on GH failed to fetch the repository, and there doesn't seem to be a way to restart it.

I can rerun that, but the fetch failure is actually definitive. Please remove v1_3_1"_990201 git tag on your fork first. We removed that tag from ruby/ruby mirror recently. Your repository might be forked before that and so still seems to have that. https://github.com/ThirteenLtda/ruby/releases/tag/v1_3_1%22_990201

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77063

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91789] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).

ruby -v changed from ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu] to trunk

I'll check appveyor out

I've `push --mirror` from the current ruby git repo to remove all dead references, and re-pushed the branch. I didn't realize it would force me to re-create the pull request, which is now https://github.com/ruby/ruby/pull/2093.

I'll make sure the AppVeyor build passes, and report here when it does.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77064

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91814] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[eregontp]

Issue #15438 has been updated by Eregon (Benoit Daloze).


Why is RUBY_THREAD_PRIORITY_MIN changed? This should be discussed in this thread.

Other implementations have to follow this change (e.g. TruffleRuby & JRuby already handle negative priorities AFAIK),
so we need a good spec to ensure in a given Ruby versions the limit is the same for all implementations.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77087

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91815] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


`RUBY_THREAD_PRIORITY_MIN` is changed to give the developer the choice to make some threads really low priority, at the cost of switching overhead. Unless I'm mistaken, the semantic of 'priority' right now is very platform-specific anyways (depends on the underlying thread scheduling implementation), so I don't see a major drive in making the min and max value uniform across all Ruby implementations.

If this indeed is a problem, I'll just switch it back to -3.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77088

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91817] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[eregontp]

Issue #15438 has been updated by Eregon (Benoit Daloze).


> the semantic of 'priority' right now is very platform-specific anyways 

The specific semantics maybe, but the range is platform-agnostic and has always been -3..3 so far.

> I don't see a major drive in making the min and max value uniform across all Ruby implementations.

I think it's important, otherwise how is Ruby code supposed to know what are the limits and which value it can use for `Thread#priority=` ?

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77089

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91821] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


> I think it's important, otherwise how is Ruby code supposed to know what are the limits and which value it can use for Thread#priority= ?

It does not have to know. It will use, say -5 because it is meaningful on MRI. The value is then clamped to -3 by e.g. JRuby where -5 is not meaningful. This is already the behavior of Thread#priority - clamp to [min, max], https://github.com/ruby/spec/pull/661 only stopped assuming that the only range is [-3, 3]

This IMO does not add any more variability than there already is between the platforms, where '-3' already maps to different behaviors.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77093

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91833] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[eregontp]

Issue #15438 has been updated by Eregon (Benoit Daloze).


> It does not have to know. It will use, say -5 because it is meaningful on MRI. The value is then clamped to -3 by e.g. JRuby where -5 is not meaningful. 

-5 would still mean "the highest priority", yes, but then e.g. -4 or -3 which starts to mean "high but not highest priority" on MRI would have a different meaning on other implementations, which I believe is undesirable.

FWIW, TruffleRuby and JRuby map -3..3 to Java 1..10:
https://github.com/oracle/truffleruby/blob/ed796f79515351f8c569306e01775c13e48876e8/src/main/ruby/core/thread.rb#L271-L276
So making the range larger would actually allow finer control on those implementations too.
setpriority(2) has 40 values on Linux.

Maybe we should use something else than integers, like a floating point number or Rational between 0 and 1, and internally map to the closest value?
I think that would be both more flexible and portable than extending the range on one side just for one implementation.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77103

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91835] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


> -5 would still mean "the highest priority", yes, but then e.g. -4 or -3 which starts to mean "high but not highest priority" on MRI would have a different meaning on other implementations, which I believe is undesirable.

-5 means "give a timeslice of 3.1ms to this thread" while the threads at 0 will have a timeslice of 100ms and 3 have 800ms. So, -5 is lowest, not highest.

I guess we'll have to agree to disagree on the desirability issue. -3 on MRI and -3 on JRuby will already cause a different behavior in the various implementations. Someone who would want to fine-tune threading behavior will already have to find which values to use on which implementation.

> Maybe we should use something else than integers, like a floating point number or Rational between 0 and 1, and internally map to the closest value?

I guess we could ... but on MRI we would still have to "guess" what are both the lowest and highest time slices we want to support *forever* and embed that in the value. Not really a very good prospect IMO.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77105

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91959] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


OK ... since it seems that everyone is camping on their own position, I have an alternative proposal: initialize the min and max priority values using environment variables on MRI, defaults to -3 and 3. Would that be alright ?

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77293

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91966] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[eregontp]

Issue #15438 has been updated by Eregon (Benoit Daloze).


> I guess we could ... but on MRI we would still have to "guess" what are both the lowest and highest time slices we want to support forever and embed that in the value. Not really a very good prospect IMO.

Does it matter what the exact timeslice is in absolute units, or what matters is how often is a high-priority Thread executed compared to a low-priority Thread?

If the first one, maybe the API should be Thread#timeslice=, but I have no idea how to map e.g. getpriority/setpriority to that (on implementations without a GIL).

> Someone who would want to fine-tune threading behavior will already have to find which values to use on which implementation.

How about exposing Thread::MIN_PRIORITY and Thread::MAX_PRIORITY? Then one could know what's the valid range of values.
I would be fine with that as an evolution mechanism (and bump MAX_PRIORITY from currently 3 to 5 on MRI).

I understand the desire to keep the same timeslice values on MRI for compatibility.
I'm trying to come with an API that is still portable and doesn't silently truncate values a user cannot easily query.

In the end, this is just my opinion. Other MRI committers, please weigh in. cc @nobu

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77299

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91977] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[sylvain.joyeux]

Issue #15438 has been updated by sylvain.joyeux (Sylvain Joyeux).


> Does it matter what the exact timeslice is in absolute units, or what matters is how often is a high-priority Thread executed compared to a low-priority Thread?

The actual underlying behavior matters, actually. If I'm using Linux scheduler, I know how to play to get what I want. On MRI, the control is the timeslice, so yes I want to know what timeslice I get.

What I really don't see is how you can expect to get a common set of values, the underlying scheduling system being always different.

I don't see any downside with exposing MIN_PRIORITY and MAX_PRIORITY. If that would be fine for you too, I'll just change the PR to reflect that.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77309

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:91985] [Ruby trunk Bug#15438] Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)

[eregontp]

Issue #15438 has been updated by Eregon (Benoit Daloze).


sylvain.joyeux (Sylvain Joyeux) wrote:
> > Does it matter what the exact timeslice is in absolute units, or what matters is how often is a high-priority Thread executed compared to a low-priority Thread?
> 
> The actual underlying behavior matters, actually. If I'm using Linux scheduler, I know how to play to get what I want. On MRI, the control is the timeslice, so yes I want to know what timeslice I get.

OK, then I think it would be worth documenting what the timeslice is on MRI based on the priority value in the docs of Thread#priority=, and mention other implementations might interpret differently.

> What I really don't see is how you can expect to get a common set of values, the underlying scheduling system being always different.

Shouldn't we be able to have e.g. a Thread with half the priority of another, regardless of the OS/Ruby implementation?

> I don't see any downside with exposing MIN_PRIORITY and MAX_PRIORITY. If that would be fine for you too, I'll just change the PR to reflect that.

I think that would be a useful addition and gives a path forward for what is the priority value range, so please do.

I'm not very familiar with scheduling priorities, so another review/opinion would definitely be welcome.

----------------------------------------
Bug #15438: Threads can't switch faster than TIME_QUANTUM_(NSEC|USEC|MSEC)
https://bugs.ruby-lang.org/issues/15438#change-77317

* Author: sylvain.joyeux (Sylvain Joyeux)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: trunk
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN
----------------------------------------
Thread#priority can be set to negative values, which when looking at the code is meant to reduce the time allocated to the thread. However, as far as I could understand in the codebase, the quantum of time is definitely hard-coded to 100ms (TIME_QUANTUM_...). This means that the "lower allocated time" would only work for threads that would often yield one way or the other (sleep, blocking calls, ...)

My projects would definitely benefit from a faster switching period. I was wondering how best to implement this ability ?

I thought of the following:

1. globally using an environment variable
2. globally using an API
3. trying to adapt dynamically, using the highest needed period
4. lowering the period when a priority lower than 0 is set, leaving it at the lower period.

Obviously (3) would seem to be the best, but I'm not sure I would be able to get it right in a decent amount of time. (4) seem to be a good trade-off between simplicity and performance (nothing changes if you never use priorities lower than 0, and if you were you basically get what you wanted).

What do you think ?

---Files--------------------------------
0001-dynamically-modify-the-timer-thread-period-to-accoun.patch (3.12 KB)
0001-2.6-fix-handling-of-negative-priorities.patch (8.43 KB)


-- 
https://bugs.ruby-lang.org/