ruby-core@ruby-lang.org archive (unofficial mirror)
 help / color / mirror / Atom feed
* [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
@ 2024-03-20 14:57 eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-04 20:03 ` [ruby-core:117442] " eightbitraptor (Matthew Valentine-House) via ruby-core
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: eightbitraptor (Matthew Valentine-House) via ruby-core @ 2024-03-20 14:57 UTC (permalink / raw)
  To: ruby-core; +Cc: eightbitraptor (Matthew Valentine-House)

Issue #20351 has been reported by eightbitraptor (Matthew Valentine-House).

----------------------------------------
Feature #20351: Optionally extract common GC routines into a DSO
https://bugs.ruby-lang.org/issues/20351

* Author: eightbitraptor (Matthew Valentine-House)
* Status: Open
----------------------------------------
[Github PR#10302](https://github.com/ruby/ruby/pull/10302)

**NOTE: This proposal does not change the default build of Ruby, and therefore
should NOT cause performance degradation for Ruby built in the usual way**

Our long term goal is to standardise Ruby's GC interface, allowing alternative
GC implementations to be used. This will be acheived by optionally building
Ruby's GC as a shared object; enabling it to be replaced at runtime using using
`LD_LIBRARY_PATH`. eg:

```
LD_LIBRARY_PATH=/custom_gc_location ruby script.rb
```

This ticket proposes the first step towards this goal. A new experimental build
option, `--enable-shared-gc`, that will compile and link a module into the built
`ruby` binary as a shared object - `miniruby` will remain statically linked to
the existing GC in all cases.


Similar methods of replacing functionality relied on by Ruby have
precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and
`free` at runtime. Although this project will be the first time a technique such
as this has been used to replace core Ruby functionality.

This flag will be marked as experimental & **disabled by default**.

[The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the
absolute minimum code required to test it's implementation (a single debug
function).

The implementation of the new build flag is based on the existing implementation
of `--enable-shared` and behaves as follows:

- `--enable-shared --enable-shared-gc`
  
  This will build both `libruby` and `librubygc` as shared objects. `ruby` will
  link dynamically to both `libruby` and `librubygc`.
  
- `--disable-shared --enable-shared-gc`

  This will build `librubygc` as a shared object, and build `libruby` as a
  static object. `libruby` will link dynamically to `librubygc` and `ruby` will
  be statically linked to `libruby`.
  
- `--disable-shared-gc`

  **This will be the default**, and when this case is true the build behaviour
  will be exactly the same as it is currently. ie. the existing Ruby GC will be
  built and linked statically into either `ruby` or `libruby.so` depending on
  the state of `--enable-shared`.
  
We are aware that there will be a small performance penalty from moving the GC
logic into a shared object, but this is an opt-in configuration turned on at
build time intended to be used by experienced users. 

Still, we anticipate that, even with this configuration turned on, this penalty
will be negligible compared the the benefit that being able to use high
performance GC algorithms will provide.

This performance penalty is also the reason that **this feature will be disabled
by default**. There will be no performance impact for anyone compiling Ruby in
the usual manner, without explicitly enabling this feature.

We have discussed this proposal with @matz who has approved our work on this
project - having a clear abstraction between the VM and the GC will enable us to
iterate faster on improvements to Ruby's existing GC.

## Motivation

In the long term we want to provide the ability to override the current Ruby GC
implementation in order to:

* Experiment with modern high-performance GC implementations, such as Immix, G1,
  LXR etc.
* Easily split-test changes to the GC, or the GC tuning, in production without
  having to rebuild Ruby
* Easily use debug builds of the GC to help identify production problems and
  bottlenecks without having to rebuild Ruby
* Encourage the academic memory management research community to consider Ruby
  for their research (the current work on [MMTk & Ruby]() is a good example of
  this).

## Future work

The initial implementation of the shared GC module in this PR is deliberately
small, and exists only for testing the build system integration.

The next steps are to identify boundaries between the GC and the VM and begin to
extract common functionality into this GC wrapper module to serve as the
foundation of our GC interface.

## Who's working on this
  
- Matt Valentine-House (@eightbitraptor) 
- Aaron Patterson (@tenderlove)
- Peter Zhu(@peterzhu2118)
- Eileen Uchitelle(@eileencodes)



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117442] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
  2024-03-20 14:57 [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO eightbitraptor (Matthew Valentine-House) via ruby-core
@ 2024-04-04 20:03 ` eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-15 20:02 ` [ruby-core:117514] " eightbitraptor (Matthew Valentine-House) via ruby-core
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: eightbitraptor (Matthew Valentine-House) via ruby-core @ 2024-04-04 20:03 UTC (permalink / raw)
  To: ruby-core; +Cc: eightbitraptor (Matthew Valentine-House)

Issue #20351 has been updated by eightbitraptor (Matthew Valentine-House).


Based on feedback from @katei and @nobu we have changed our approach to this project.

An updated PR can be found here: [[GH #10456]](https://github.com/ruby/ruby/pull/10456)

Instead of building the Ruby GC as a shared object in order to override it using `LD_LIBRARY_PATH`/`LD_PRELOAD` et al, and after experimenting with some alternative approaches we have decided to use an approach based around the `dlopen` wrappers provided in `dln.c`. This provides several benefits that our existing implementation does not.

- It does not require integration of another shared object into the build system. As pointed out by @katei, this is a complex and error-prone task, with added complexity implications for committers building and testing new features on Ruby, as well as an explosion of new CI tasks required to test all variants.
- It doesn't expose the GC as a shared object, with the potentially misleading consequence of users assuming they can link against the Ruby GC independently; as they currently can with `libruby` 
- It runs on more platforms than our initial approach which was targeted specifically at Linux and MacOS. The use of existing functionality in `dln.c` should allow development of this feature on any platform that supports C extensions.

The approach taken here is as follows:

- Enable this feature by configuring with `--with-shared-gc`
- When enabled, Ruby will check for the presence of the environment variable `RUBY_GC_LIBRARY_PATH`
- If that path exists and points to a valid shared object
  - Open the shared object (using `dln_open`), and map the `Init_GC` function to a function map
  - call `Init_GC` as part of `rb_objspace_alloc` when initialising the GC
- If that path does not point to a valid shared object
  - Degrade gracefully to initialising the existing Ruby objspace and GC
  
When Ruby is configured without the `--with-shared-gc` flag, no extra code is compiled and so existing behaviour is maintained without any performance penalty.

Enabling this feature, but not populating `RUBY_GC_LIBRARY_PATH` will incur a small performance penalty due to the overhead of using `getenv` to check for the environment variable.

The use of `dln.c` rather than using `dlopen` directly, should enable this feature to be supported on all platforms that support loading shared libraries, rather than the initial implementation, which explicitly only supported Linux and MacOS.

----------------------------------------
Feature #20351: Optionally extract common GC routines into a DSO
https://bugs.ruby-lang.org/issues/20351#change-107823

* Author: eightbitraptor (Matthew Valentine-House)
* Status: Open
----------------------------------------
[Github PR#10302](https://github.com/ruby/ruby/pull/10302)

**NOTE: This proposal does not change the default build of Ruby, and therefore
should NOT cause performance degradation for Ruby built in the usual way**

Our long term goal is to standardise Ruby's GC interface, allowing alternative
GC implementations to be used. This will be acheived by optionally building
Ruby's GC as a shared object; enabling it to be replaced at runtime using using
`LD_LIBRARY_PATH`. eg:

```
LD_LIBRARY_PATH=/custom_gc_location ruby script.rb
```

This ticket proposes the first step towards this goal. A new experimental build
option, `--enable-shared-gc`, that will compile and link a module into the built
`ruby` binary as a shared object - `miniruby` will remain statically linked to
the existing GC in all cases.


Similar methods of replacing functionality relied on by Ruby have
precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and
`free` at runtime. Although this project will be the first time a technique such
as this has been used to replace core Ruby functionality.

This flag will be marked as experimental & **disabled by default**.

[The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the
absolute minimum code required to test it's implementation (a single debug
function).

The implementation of the new build flag is based on the existing implementation
of `--enable-shared` and behaves as follows:

- `--enable-shared --enable-shared-gc`
  
  This will build both `libruby` and `librubygc` as shared objects. `ruby` will
  link dynamically to both `libruby` and `librubygc`.
  
- `--disable-shared --enable-shared-gc`

  This will build `librubygc` as a shared object, and build `libruby` as a
  static object. `libruby` will link dynamically to `librubygc` and `ruby` will
  be statically linked to `libruby`.
  
- `--disable-shared-gc`

  **This will be the default**, and when this case is true the build behaviour
  will be exactly the same as it is currently. ie. the existing Ruby GC will be
  built and linked statically into either `ruby` or `libruby.so` depending on
  the state of `--enable-shared`.
  
We are aware that there will be a small performance penalty from moving the GC
logic into a shared object, but this is an opt-in configuration turned on at
build time intended to be used by experienced users. 

Still, we anticipate that, even with this configuration turned on, this penalty
will be negligible compared the the benefit that being able to use high
performance GC algorithms will provide.

This performance penalty is also the reason that **this feature will be disabled
by default**. There will be no performance impact for anyone compiling Ruby in
the usual manner, without explicitly enabling this feature.

We have discussed this proposal with @matz who has approved our work on this
project - having a clear abstraction between the VM and the GC will enable us to
iterate faster on improvements to Ruby's existing GC.

## Motivation

In the long term we want to provide the ability to override the current Ruby GC
implementation in order to:

* Experiment with modern high-performance GC implementations, such as Immix, G1,
  LXR etc.
* Easily split-test changes to the GC, or the GC tuning, in production without
  having to rebuild Ruby
* Easily use debug builds of the GC to help identify production problems and
  bottlenecks without having to rebuild Ruby
* Encourage the academic memory management research community to consider Ruby
  for their research (the current work on [MMTk & Ruby](https://github.com/mmtk/mmtk-ruby) is a good example of
  this).

## Future work

The initial implementation of the shared GC module in this PR is deliberately
small, and exists only for testing the build system integration.

The next steps are to identify boundaries between the GC and the VM and begin to
extract common functionality into this GC wrapper module to serve as the
foundation of our GC interface.

## Who's working on this
  
- @eightbitraptor
- @tenderlovemaking 
- @peterzhu2118
- @eileencodes



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117514] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
  2024-03-20 14:57 [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-04 20:03 ` [ruby-core:117442] " eightbitraptor (Matthew Valentine-House) via ruby-core
@ 2024-04-15 20:02 ` eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-16  1:01 ` [ruby-core:117520] " ko1 (Koichi Sasada) via ruby-core
  2024-04-16  8:41 ` [ruby-core:117527] " eightbitraptor (Matthew Valentine-House) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: eightbitraptor (Matthew Valentine-House) via ruby-core @ 2024-04-15 20:02 UTC (permalink / raw)
  To: ruby-core; +Cc: eightbitraptor (Matthew Valentine-House)

Issue #20351 has been updated by eightbitraptor (Matthew Valentine-House).


Closed by https://github.com/ruby/ruby/pull/10456

----------------------------------------
Feature #20351: Optionally extract common GC routines into a DSO
https://bugs.ruby-lang.org/issues/20351#change-107903

* Author: eightbitraptor (Matthew Valentine-House)
* Status: Open
----------------------------------------
**UPDATE: Based on feedback on the original PR (thank you @katei and @nobu) we have
changed our approach to this project.**

**An updated PR can be found here: [[GH #10456]](https://github.com/ruby/ruby/pull/10456)**

---

~~[Github PR#10302](https://github.com/ruby/ruby/pull/10302)~~

**NOTE: This proposal does not change the default build of Ruby, and therefore
should NOT cause performance degradation for Ruby built in the usual way**

Our long term goal is to standardise Ruby's GC interface, allowing alternative
GC implementations to be used. This will be acheived by optionally building
Ruby's GC as a shared object; enabling it to be replaced at runtime using using
`LD_LIBRARY_PATH`. eg:

```
LD_LIBRARY_PATH=/custom_gc_location ruby script.rb
```

This ticket proposes the first step towards this goal. A new experimental build
option, `--enable-shared-gc`, that will compile and link a module into the built
`ruby` binary as a shared object - `miniruby` will remain statically linked to
the existing GC in all cases.


Similar methods of replacing functionality relied on by Ruby have
precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and
`free` at runtime. Although this project will be the first time a technique such
as this has been used to replace core Ruby functionality.

This flag will be marked as experimental & **disabled by default**.

[The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the
absolute minimum code required to test it's implementation (a single debug
function).

The implementation of the new build flag is based on the existing implementation
of `--enable-shared` and behaves as follows:

- `--enable-shared --enable-shared-gc`
  
  This will build both `libruby` and `librubygc` as shared objects. `ruby` will
  link dynamically to both `libruby` and `librubygc`.
  
- `--disable-shared --enable-shared-gc`

  This will build `librubygc` as a shared object, and build `libruby` as a
  static object. `libruby` will link dynamically to `librubygc` and `ruby` will
  be statically linked to `libruby`.
  
- `--disable-shared-gc`

  **This will be the default**, and when this case is true the build behaviour
  will be exactly the same as it is currently. ie. the existing Ruby GC will be
  built and linked statically into either `ruby` or `libruby.so` depending on
  the state of `--enable-shared`.
  
We are aware that there will be a small performance penalty from moving the GC
logic into a shared object, but this is an opt-in configuration turned on at
build time intended to be used by experienced users. 

Still, we anticipate that, even with this configuration turned on, this penalty
will be negligible compared the the benefit that being able to use high
performance GC algorithms will provide.

This performance penalty is also the reason that **this feature will be disabled
by default**. There will be no performance impact for anyone compiling Ruby in
the usual manner, without explicitly enabling this feature.

We have discussed this proposal with @matz who has approved our work on this
project - having a clear abstraction between the VM and the GC will enable us to
iterate faster on improvements to Ruby's existing GC.

## Motivation

In the long term we want to provide the ability to override the current Ruby GC
implementation in order to:

* Experiment with modern high-performance GC implementations, such as Immix, G1,
  LXR etc.
* Easily split-test changes to the GC, or the GC tuning, in production without
  having to rebuild Ruby
* Easily use debug builds of the GC to help identify production problems and
  bottlenecks without having to rebuild Ruby
* Encourage the academic memory management research community to consider Ruby
  for their research (the current work on [MMTk & Ruby](https://github.com/mmtk/mmtk-ruby) is a good example of
  this).

## Future work

The initial implementation of the shared GC module in this PR is deliberately
small, and exists only for testing the build system integration.

The next steps are to identify boundaries between the GC and the VM and begin to
extract common functionality into this GC wrapper module to serve as the
foundation of our GC interface.

## Who's working on this
  
- @eightbitraptor
- @tenderlovemaking 
- @peterzhu2118
- @eileencodes



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117520] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
  2024-03-20 14:57 [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-04 20:03 ` [ruby-core:117442] " eightbitraptor (Matthew Valentine-House) via ruby-core
  2024-04-15 20:02 ` [ruby-core:117514] " eightbitraptor (Matthew Valentine-House) via ruby-core
@ 2024-04-16  1:01 ` ko1 (Koichi Sasada) via ruby-core
  2024-04-16  8:41 ` [ruby-core:117527] " eightbitraptor (Matthew Valentine-House) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: ko1 (Koichi Sasada) via ruby-core @ 2024-04-16  1:01 UTC (permalink / raw)
  To: ruby-core; +Cc: ko1 (Koichi Sasada)

Issue #20351 has been updated by ko1 (Koichi Sasada).


Please note that we are working on ractor local GC and I'm not sure we can fix "GC interface".

----------------------------------------
Feature #20351: Optionally extract common GC routines into a DSO
https://bugs.ruby-lang.org/issues/20351#change-107909

* Author: eightbitraptor (Matthew Valentine-House)
* Status: Closed
----------------------------------------
**UPDATE: Based on feedback on the original PR (thank you @katei and @nobu) we have
changed our approach to this project.**

**An updated PR can be found here: [[GH #10456]](https://github.com/ruby/ruby/pull/10456)**

---

~~[Github PR#10302](https://github.com/ruby/ruby/pull/10302)~~

**NOTE: This proposal does not change the default build of Ruby, and therefore
should NOT cause performance degradation for Ruby built in the usual way**

Our long term goal is to standardise Ruby's GC interface, allowing alternative
GC implementations to be used. This will be acheived by optionally building
Ruby's GC as a shared object; enabling it to be replaced at runtime using using
`LD_LIBRARY_PATH`. eg:

```
LD_LIBRARY_PATH=/custom_gc_location ruby script.rb
```

This ticket proposes the first step towards this goal. A new experimental build
option, `--enable-shared-gc`, that will compile and link a module into the built
`ruby` binary as a shared object - `miniruby` will remain statically linked to
the existing GC in all cases.


Similar methods of replacing functionality relied on by Ruby have
precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and
`free` at runtime. Although this project will be the first time a technique such
as this has been used to replace core Ruby functionality.

This flag will be marked as experimental & **disabled by default**.

[The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the
absolute minimum code required to test it's implementation (a single debug
function).

The implementation of the new build flag is based on the existing implementation
of `--enable-shared` and behaves as follows:

- `--enable-shared --enable-shared-gc`
  
  This will build both `libruby` and `librubygc` as shared objects. `ruby` will
  link dynamically to both `libruby` and `librubygc`.
  
- `--disable-shared --enable-shared-gc`

  This will build `librubygc` as a shared object, and build `libruby` as a
  static object. `libruby` will link dynamically to `librubygc` and `ruby` will
  be statically linked to `libruby`.
  
- `--disable-shared-gc`

  **This will be the default**, and when this case is true the build behaviour
  will be exactly the same as it is currently. ie. the existing Ruby GC will be
  built and linked statically into either `ruby` or `libruby.so` depending on
  the state of `--enable-shared`.
  
We are aware that there will be a small performance penalty from moving the GC
logic into a shared object, but this is an opt-in configuration turned on at
build time intended to be used by experienced users. 

Still, we anticipate that, even with this configuration turned on, this penalty
will be negligible compared the the benefit that being able to use high
performance GC algorithms will provide.

This performance penalty is also the reason that **this feature will be disabled
by default**. There will be no performance impact for anyone compiling Ruby in
the usual manner, without explicitly enabling this feature.

We have discussed this proposal with @matz who has approved our work on this
project - having a clear abstraction between the VM and the GC will enable us to
iterate faster on improvements to Ruby's existing GC.

## Motivation

In the long term we want to provide the ability to override the current Ruby GC
implementation in order to:

* Experiment with modern high-performance GC implementations, such as Immix, G1,
  LXR etc.
* Easily split-test changes to the GC, or the GC tuning, in production without
  having to rebuild Ruby
* Easily use debug builds of the GC to help identify production problems and
  bottlenecks without having to rebuild Ruby
* Encourage the academic memory management research community to consider Ruby
  for their research (the current work on [MMTk & Ruby](https://github.com/mmtk/mmtk-ruby) is a good example of
  this).

## Future work

The initial implementation of the shared GC module in this PR is deliberately
small, and exists only for testing the build system integration.

The next steps are to identify boundaries between the GC and the VM and begin to
extract common functionality into this GC wrapper module to serve as the
foundation of our GC interface.

## Who's working on this
  
- @eightbitraptor
- @tenderlovemaking 
- @peterzhu2118
- @eileencodes



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [ruby-core:117527] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO
  2024-03-20 14:57 [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO eightbitraptor (Matthew Valentine-House) via ruby-core
                   ` (2 preceding siblings ...)
  2024-04-16  1:01 ` [ruby-core:117520] " ko1 (Koichi Sasada) via ruby-core
@ 2024-04-16  8:41 ` eightbitraptor (Matthew Valentine-House) via ruby-core
  3 siblings, 0 replies; 5+ messages in thread
From: eightbitraptor (Matthew Valentine-House) via ruby-core @ 2024-04-16  8:41 UTC (permalink / raw)
  To: ruby-core; +Cc: eightbitraptor (Matthew Valentine-House)

Issue #20351 has been updated by eightbitraptor (Matthew Valentine-House).


Do you have an idea of a timeline for Ractor local GC? Is there an open feature request I can read (I've searched on Redmine and I can't find one).

It's hard for us to assess any collisions between this work and Ractor local GC at the current time as I'm unsure about the implementation details. 

Our intention is to allow the GC to be overloaded in such a way that multiple heaps can be initialised and managed independently, in isolation from each other or the VM, so potentially each Ractor could use this feature to manage it's own heap.

To be clear this feature request and the associated PR are only providing some of the base mechanism to allow us to experiment. We have not committed to an approach at this stage.

With our current knowledge I believe it is possible to accommodate per-ractor GC and we will be careful to think about this in our design. In order to do this it would be very useful to have some guidance on the current per-ractor GC work.

----------------------------------------
Feature #20351: Optionally extract common GC routines into a DSO
https://bugs.ruby-lang.org/issues/20351#change-107917

* Author: eightbitraptor (Matthew Valentine-House)
* Status: Closed
----------------------------------------
**UPDATE: Based on feedback on the original PR (thank you @katei and @nobu) we have
changed our approach to this project.**

**An updated PR can be found here: [[GH #10456]](https://github.com/ruby/ruby/pull/10456)**

---

~~[Github PR#10302](https://github.com/ruby/ruby/pull/10302)~~

**NOTE: This proposal does not change the default build of Ruby, and therefore
should NOT cause performance degradation for Ruby built in the usual way**

Our long term goal is to standardise Ruby's GC interface, allowing alternative
GC implementations to be used. This will be acheived by optionally building
Ruby's GC as a shared object; enabling it to be replaced at runtime using using
`LD_LIBRARY_PATH`. eg:

```
LD_LIBRARY_PATH=/custom_gc_location ruby script.rb
```

This ticket proposes the first step towards this goal. A new experimental build
option, `--enable-shared-gc`, that will compile and link a module into the built
`ruby` binary as a shared object - `miniruby` will remain statically linked to
the existing GC in all cases.


Similar methods of replacing functionality relied on by Ruby have
precedent. `jemalloc` uses `LD_PRELOAD` to replace `glibc` provided `malloc` and
`free` at runtime. Although this project will be the first time a technique such
as this has been used to replace core Ruby functionality.

This flag will be marked as experimental & **disabled by default**.

[The PR linked from this ticket](https://github.com/ruby/ruby/pull/10302) implements the new build flag, along with the
absolute minimum code required to test it's implementation (a single debug
function).

The implementation of the new build flag is based on the existing implementation
of `--enable-shared` and behaves as follows:

- `--enable-shared --enable-shared-gc`
  
  This will build both `libruby` and `librubygc` as shared objects. `ruby` will
  link dynamically to both `libruby` and `librubygc`.
  
- `--disable-shared --enable-shared-gc`

  This will build `librubygc` as a shared object, and build `libruby` as a
  static object. `libruby` will link dynamically to `librubygc` and `ruby` will
  be statically linked to `libruby`.
  
- `--disable-shared-gc`

  **This will be the default**, and when this case is true the build behaviour
  will be exactly the same as it is currently. ie. the existing Ruby GC will be
  built and linked statically into either `ruby` or `libruby.so` depending on
  the state of `--enable-shared`.
  
We are aware that there will be a small performance penalty from moving the GC
logic into a shared object, but this is an opt-in configuration turned on at
build time intended to be used by experienced users. 

Still, we anticipate that, even with this configuration turned on, this penalty
will be negligible compared the the benefit that being able to use high
performance GC algorithms will provide.

This performance penalty is also the reason that **this feature will be disabled
by default**. There will be no performance impact for anyone compiling Ruby in
the usual manner, without explicitly enabling this feature.

We have discussed this proposal with @matz who has approved our work on this
project - having a clear abstraction between the VM and the GC will enable us to
iterate faster on improvements to Ruby's existing GC.

## Motivation

In the long term we want to provide the ability to override the current Ruby GC
implementation in order to:

* Experiment with modern high-performance GC implementations, such as Immix, G1,
  LXR etc.
* Easily split-test changes to the GC, or the GC tuning, in production without
  having to rebuild Ruby
* Easily use debug builds of the GC to help identify production problems and
  bottlenecks without having to rebuild Ruby
* Encourage the academic memory management research community to consider Ruby
  for their research (the current work on [MMTk & Ruby](https://github.com/mmtk/mmtk-ruby) is a good example of
  this).

## Future work

The initial implementation of the shared GC module in this PR is deliberately
small, and exists only for testing the build system integration.

The next steps are to identify boundaries between the GC and the VM and begin to
extract common functionality into this GC wrapper module to serve as the
foundation of our GC interface.

## Who's working on this
  
- @eightbitraptor
- @tenderlovemaking 
- @peterzhu2118
- @eileencodes



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2024-04-16  8:41 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-20 14:57 [ruby-core:117263] [Ruby master Feature#20351] Optionally extract common GC routines into a DSO eightbitraptor (Matthew Valentine-House) via ruby-core
2024-04-04 20:03 ` [ruby-core:117442] " eightbitraptor (Matthew Valentine-House) via ruby-core
2024-04-15 20:02 ` [ruby-core:117514] " eightbitraptor (Matthew Valentine-House) via ruby-core
2024-04-16  1:01 ` [ruby-core:117520] " ko1 (Koichi Sasada) via ruby-core
2024-04-16  8:41 ` [ruby-core:117527] " eightbitraptor (Matthew Valentine-House) via ruby-core

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).