[ruby-core:104226] [Ruby master Feature#10320] require into module

[ruby-core:104226] [Ruby master Feature#10320] require into module

[j.ruby-lang]

Issue #10320 has been updated by jaesharp (J Lynn).


I'd like to note that there exists a gem called modules ( https://rubygems.org/gems/modules ) which uses `Kernel#load` with the `wrap=true` option in order to implement a module import/export resolution system similar in nature to the one described here and to node.js's commonjs module system semantics. Perhaps this is sufficient to meet people's needs, if brought in?

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-92413

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:104227] [Ruby master Feature#10320] require into module

[branzeanu.aurel]

Issue #10320 has been updated by texpert (Aurel Branzeanu).


jaesharp (J Lynn) wrote in #note-10:
> I'd like to note that there exists a gem called modules ( https://rubygems.org/gems/modules ) which uses `Kernel#load` with the `wrap=true` option in order to implement a module import/export resolution system similar in nature to the one described here and to node.js's commonjs module system semantics. Perhaps this is sufficient to meet people's needs, if brought in?

And another similar gem is modulation ( https://rubygems.org/gems/modulation/versions/0.25 )

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-92414

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:104228] [Ruby master Feature#10320] require into module

[j.ruby-lang]

Issue #10320 has been updated by jaesharp (J Lynn).


texpert (Aurel Branzeanu) wrote in #note-11:
> And another similar gem is modulation ( https://rubygems.org/gems/modulation/versions/0.25 )

It looks much more developed and actively maintained! Thank you! :)

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-92415

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109898] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


The `wrap` option to `load` has recently been expanded to allow passing a module instead of a boolean (https://bugs.ruby-lang.org/issues/6210). This opens the doors to new approaches to this problem.

I think there is a huge opportunity to improve Ruby, and particularly to improve its "namespace hygiene", in the way JS module systems work in Javascript.

Unlike others I've seen implemented in Ruby (including the [modules](https://github.com/lambdabaa/modules) gem, mentioned above), I'd like this to work not only for code that explicitly enables it (with `export`, etc.) but also (and mainly) for code (particularly gem code) that does not.

I've created a gem called [Im](https://github.com/shioyama/im), which depends on a few patches in [this Ruby branch](https://github.com/shioyama/ruby/tree/import_modules) ([c43870](https://github.com/shioyama/ruby/commit/c4387043437b0851306ef199f9417be609c98827) is the main one, plus the bug fix in https://bugs.ruby-lang.org/issues/18960). My goal here is to load _existing gems_, including some of the ones we use often. I want to focus on something concrete which would potentially immediately impact how gems use the global namespace.

With this gem and the patches provided, you can e.g. load `activemodel` and use it in this way:

```ruby
require "im"
extend Im # adds `import` method

mod = import "activemodel"
```

At this point, `ActiveModel` has been loaded but it is not in the global namespace, and can only be found in `mod`:

```ruby
ActiveModel
#=> uninitialized constant ActiveModel (NameError)

mod::ActiveModel
#=> ActiveModel
```

What is very interesting to me is that, _using Ruby's built-in temporary/permanent naming functionality_, we can alias either all of the gem or parts of it.

So for example, naming the module immediately names everything inside of it:

```ruby
MyRails = mod
```

Now we have `MyRails::ActiveModel` (but no `::ActiveModel`).


You can confirm with the example in the gem's readme that this namespaced ActiveModel works (at least for simple stuff, haven't tested more thoroughly yet).

There are a few things I do to make this work:
1. In the Ruby patch, I make the `wrap` argument to `load` apply to further `require`s inside the loaded script. This is actually as simple as removing (actually commenting out) a line.
2. Apply the `top_wrapper` created in `load` to native extensions (specifically, `rb_define_class` & `rb_define_global_const`)
3. Make named modules/classes under an anonymous namespace return their name minus the namespace as their temporary name when called with `name`. This is necessary to use gems like Rails which use `name` to load files, etc.
4. Resolve top-level references (`::Foo`) when loaded with `wrap` to the top of the module namespace, rather than the "absolute" top. For now I've done this in the gem using `const_missing`, but I intend on moving this to the Ruby patch.

With the changes above, the gem is able to setup a registry, patch `Kernel#require`, `Modul#autoload` etc and make this all work (sort of).

To make this all work, I use a lot of aliasing constants from one module namespace to another. This actually seems to work pretty well! So for example, code like this (similar to ActiveSupport):

```ruby
class NilClass
  def to_s
    # ...
  end
end
```

would define a _new_ class under the module namespace. So I alias `mod::NilClass` and other predefined constants to `NilClass`, which then makes this work.

What is amazing to me is _how little is required to make this work_.

Obviously this is very much a proof of concept at this point (and I want to emphasize that). But if you look at the changes to Ruby, they only impact code that uses the `wrap` option to `load`, and there are only a few of them. Also, to me at least, they feel quite natural, since they are simply extending a concept that already exists (`top_wrapper`) to other places where it is not yet applied.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99141

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109915] [Ruby master Feature#10320] require into module

[Eregon (Benoit Daloze)]

Issue #10320 has been updated by Eregon (Benoit Daloze).


This NilClass definition, even if reassigning global ::NilClass doesn't have any effect on nil though, isn't it?
Or do you actually define mod::NilClass = NilClass before loading the ActiveSupport files?

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99161

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109916] [Ruby master Feature#10320] require into module

[shyouhei (Shyouhei Urabe)]

Issue #10320 has been updated by shyouhei (Shyouhei Urabe).


Pure-ruby codes could perhaps be loaded multiple times side-by-side, but the problem is a DLL.
Loading ActiveRecord::Base won't work because when it tries to dynamic-link libpq.so or libmysqlclient.so or whatever, that can already be loaded under another namespace; then fails.
This is not only about rb_define_method etc.  We cannot control how DLLs are loaded by the operating system.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99162

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109918] [Ruby master Feature#10320] require into module

[nobu (Nobuyoshi Nakada)]

Issue #10320 has been updated by nobu (Nobuyoshi Nakada).


It is declared that `dlopen` will not fail in such cases.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/dlopen.html
> Only a single copy of an executable object file shall be brought into the address space, even if `dlopen()` is invoked multiple times in reference to the executable object file, and even if different pathnames are used to reference the executable object file.

---------

`LoadLibrary` on Windows is similar.
https://docs.microsoft.com/en-us/cpp/build/loadlibrary-and-afxloadlibrary?view=msvc-170
> If the call to `LoadLibrary` specifies a DLL module that is already mapped into the address space of the calling process, the function returns a handle of the DLL and increments the reference count of the module.

---------

I couldn't find explicit descriptions for `load` on AIX.
https://www.ibm.com/docs/en/aix/7.3?topic=l-load-loadandinit-subroutines

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99165

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109923] [Ruby master Feature#10320] require into module

[byroot (Jean Boussier)]

Issue #10320 has been updated by byroot (Jean Boussier).


> Loading ActiveRecord::Base won't work because when it tries to dynamic-link libpq.so or libmysqlclient.so or whatever, that can already be loaded under another namespace; then fails.

An orthogonal question is whether loading two versions of Active Record should load two versions of all the dependencies of Active Record?

I have only quickly skimmed `Im`, so I may say innacurate things, but it seems to me that what would be desirable would be to load a "namespace" in isolation, but not necessarily its dependencies as well.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99171

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109925] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


> I have only quickly skimmed Im, so I may say innacurate things, but it seems to me that what would be desirable would be to load a "namespace" in isolation, but not necessarily its dependencies as well.

`Im` uses a registry to track files that have been loaded by their filenames, using TracePoint to also track which classes/modules were created in the process. It also patches `require`, so that if somewhere else in another import, the same file is required, the constants can all be aliased, allowing them to "share" the same dependencies.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99172

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109926] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


> This NilClass definition, even if reassigning global ::NilClass doesn't have any effect on nil though, isn't it?
Or do you actually define mod::NilClass = NilClass before loading the ActiveSupport files?

Yes, `mod::NilClass = NilClass` is assigned in the module before passing it to the first `load`, so when loading ActiveSupport core extensions AS sees `mod::NilClass` and this simply points to `::NilClass`.

You can confirm it works:

```ruby
nil.blank?
#  undefined method `blank?' for nil:NilClass (NoMethodError)

require "im"
extend Im

mod = import "active_support"
#=> <#Im::Import root: active_support>

ActiveSupport
# `const_missing': uninitialized constant ActiveSupport (NameError)


mod::ActiveSupport
#=> ActiveSupport

nil.blank?
#=> true
```
nil.blank?
#=> true
```

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99173

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109927] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


@shyouhei

> This is not only about rb_define_method etc. We cannot control how DLLs are loaded by the operating system.

Thanks! This is the kind of thing I felt I must be missing.

So it seems there are a relatively small number of these. All of `PG` and whatever it defines at toplevel could be made global and aliased from namespaces, which I think would avoid that problem.

The reason why I focused on `rb_define_method` etc is that without that change libraries like `CGI` get split, the Ruby part being under the isolated namespace and the other being toplevel. It's not a problem to have some stuff (like those using DLL) at toplevel, but having a library split between both would definitely be a problem.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99174

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109946] [Ruby master Feature#10320] require into module

[shyouhei (Shyouhei Urabe)]

Issue #10320 has been updated by shyouhei (Shyouhei Urabe).


May I ask someone the problem this ticket is currently trying to address? I’m confused.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99195

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109955] [Ruby master Feature#10320] require into module

[duerst]

Issue #10320 has been updated by duerst (Martin Dürst).


shyouhei (Shyouhei Urabe) wrote in #note-21:
> May I ask someone the problem this ticket is currently trying to address? I’m confused.

Same question here.

My (I hope average Ruby progarmmer) understanding of how gems/modules/namespaces are supposed to work currently is as follows (overview):
- Each gem uses a (top) module, where the module name is (modulo some case/... changes) the same as the gem name.
- Each gem puts its constants (which included classes and modules) into that gem's (top) module.
- rubygems.org makes sure that each gem name can only be used for one gem, and thus module names used by different gems are different.

The above is mostly a social contract on how to use modules in gems, but my understanding is that it is widely understood and followed. I'm sure there are exceptions, but I'd guess they happen mostly in toy "gems" written by beginners.

My guess at reasons for this proposal are the following (but I would of course like to know the real reasons):
1) Namespaces are a precious resource, and the less we sit on it, the better, even if practically, there may not be much of a problem.
2) Some people are used to how modules work in JavaScript, and want the same in Ruby.
3) Different modules may require the same modules but e.g. with different versions. (I didn't find any discussion of versions above, however, and the discussion of "using the same module by aliasing when it's included more than once" seems to indicate that different versions are not relevant here. Also, versioning is mostly bundler's job.
4) Companies (but not only companies) using Ruby have internal libraries with modules. They don't want to squat on a gem name (doing so may reveal company internals including business plans), but they don't want to risk a future name conflict.

The above are only guesses. There may by more than one reason. But I think we should make it/them as explicit as possible.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99206

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109956] [Ruby master Feature#10320] require into module

[byroot (Jean Boussier)]

Issue #10320 has been updated by byroot (Jean Boussier).


@duerst / @shyouhei 

The main goal is to avoid accidental dependency. By not exposing some namespaces globally, you can force developers to have to declare what they depend on, which makes them realize they're about to depend on something they shouldn't.


You can see https://github.com/Shopify/packwerk as prior art trying to enforce boundaries using static analysis.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99207

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109959] [Ruby master Feature#10320] require into module

[vo.x (Vit Ondruch)]

Issue #10320 has been updated by vo.x (Vit Ondruch).


I'd love to see if RubyGems/Bundler could stop vendoring packages:

https://github.com/rubygems/rubygems/tree/master/lib/rubygems
https://github.com/rubygems/rubygems/tree/master/bundler/lib/bundler/vendor

After all, Ruby could ship multiple versions of some library if old version is needed for whatever reason.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99209

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109960] [Ruby master Feature#10320] require into module

[retro]

Issue #10320 has been updated by retro (Josef Šimánek).


vo.x (Vit Ondruch) wrote in #note-24:
> I'd love to see if RubyGems/Bundler could stop vendoring packages:
> 
> https://github.com/rubygems/rubygems/tree/master/lib/rubygems
> https://github.com/rubygems/rubygems/tree/master/bundler/lib/bundler/vendor
> 
> After all, Ruby could ship multiple versions of some library if old version is needed for whatever reason.

I had initially same idea looking at https://bugs.ruby-lang.org/issues/6210, but it is not this easy actually (mostly for rubygems/bundler updates).

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99211

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109973] [Ruby master Feature#10320] require into module

[mame (Yusuke Endoh)]

Issue #10320 has been updated by mame (Yusuke Endoh).


The proposed PR is simple, small, and appears to have few performance or compatibility issues. So I am basically positive about this proposal.

> The main goal is to avoid accidental dependency.

I guess that the ultimate goal is to modularize the monolith to microservices, and that this proposal is for the intermediate stage (i.e., to modularize the monolith in a process). Am I right? It is not so obvious to me that this intermediate step would be useful, maybe because I don't have enough experience in monolith development :-)

> Resolve top-level references (::Foo) when loaded with wrap to the top of the module namespace, rather than the "absolute" top. For now I've done this in the gem using const_missing, but I intend on moving this to the Ruby patch.

This approach looks not very robust. If there is a constant `Foo` defined in the top-level, I think it does not work.

```
# foo.rb
A = :foo

p ::A
```

```
# main.rb
A = :main

load "foo.rb", Module.new #=> expect: :foo, Im hack: :main
```

More dedicated support in the Ruby core side would be necessary, I think. But I am curious about how much the change brings performance degeneration.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99223

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109974] [Ruby master Feature#10320] require into module

[byroot (Jean Boussier)]

Issue #10320 has been updated by byroot (Jean Boussier).


> I guess that the ultimate goal is to modularize the monolith to microservices, and that this proposal is for the intermediate stage (i.e., to modularize the monolith in a process). Am I right?

No, we have absolutely no intention to go with microservices, quite the opposite. The goal is to modularize in process so that you can more easily enforce that certain areas are decoupled from others without having to deal with the headaches of network calls.

> This approach looks not very robust. If there is a constant Foo defined in the top-level, I think it does not work.

Yeah, IMHO it uses too many fragile monkey patches and Tracepoint hooks to approximate the desired result. I think such a feature would need to be baked in Ruby itself with probably a keyword etc. But in the meantime "Im" is an interesting experimentation ground.

Also if such first class feature was to be designed, I think the Python import system would be a better model than NodeJS's `require()`.


----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99224

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109979] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


@mame 

> This approach looks not very robust. If there is a constant Foo defined in the top-level, I think it does not work.

Yes absolutely, this is the problem with `const_missing`. So this would ultimately need something at the language level.

To be clear, my goal is that Ruby implements the parts of this problem which are not implementable in gem code, and lets a gem like Im do the rest (similar to Zeitwerk's relationship to `autoload`).

@byroot

> IMHO it uses too many fragile monkey patches and Tracepoint hooks to approximate the desired result. I think such a feature would need to be baked in Ruby itself with probably a keyword etc. But in the meantime "Im" is an interesting experimentation ground.

Yes, I'm using whatever I can without changing Ruby too much to make things work. Also, it's of course a WIP.

But ultimately, my idea is for the gem to motivate the changes that are needed at the language level. I find the Tracepoint hook actually works ok (and Zeitwerk also uses Tracepoint), but the `const_missing` is definitely a temporary hack.

@shyouhei

> May I ask someone the problem this ticket is currently trying to address? I’m confused.

Matz talked about the problem ("better packages") in his [2021 Euruko talk](https://youtu.be/Dp12a3KGNFw?t=2956).

From my point of view, there are two major problems the proposal here would solve.

The first, namespace pollution, is the problem that gems can park themselves anywhere they like in the global namespace. Of course most gems are good citizens and limit themselves to a single module constant, but the fact that as a consumer of code (_any_ code) you have to _trust_ third parties not to pollute the shared namespace is, IMO, a huge problem.

The proposal here would make the consumer ("importer") of such code have full control over its position in the global namespace. As a user of a gem, you could name it whatever you want, or even pick and choose parts of the gem and put them wherever you want in your namespace (e.g. `AS = mod::ActiveSupport`). Of course this control is not _absolute_, since core top-level constants would be shared (so e.g. activesupport would still be able to patch shared classes) -- so this is not entirely without side-effects -- but nonetheless it would be a huge improvement over what we currently have.

Note this also would open the door to things Ruby has never been able to do, e.g. load two versions of the same gem in the same namespace. Useful? I don't know, but if it's not hard to do, I think Ruby should make it _possible_.

The second problem is the one that Jean mentioned. [Packwerk](https://github.com/Shopify/packwerk) has become quite popular among companies with large Rails monoliths as a way to isolate parts of the application by "enforcing" boundaries. But Packwerk adds yet another level of organization ("packages") on top of many others we already have ("gems", "modules", in the case of Rails also "engines", etc.)

What is to me so special about the idea here is that it would allow you to isolate parts of an application, while also sharing code as necessary.

e.g. `Shop` is a class that is used in many places in the Shopify monolith. We can "share" that class without sharing _all_ shop-related code like this:

```ruby
platform = import "components/platform"
shop_identity = import "components/shop_identity"

platform::Shop = shop_identity::Shop
```

To me, this is a very elegant way of representing dependencies between components of an application. Rather than just give a part of the code _full_ access to another part of the code (and vice versa), you just share whatever parts are needed, and not the rest.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99228

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109981] [Ruby master Feature#10320] require into module

[mame (Yusuke Endoh)]

Issue #10320 has been updated by mame (Yusuke Endoh).


byroot (Jean Boussier) wrote in #note-27:
> No, we have absolutely no intention to go with microservices, quite the opposite. The goal is to modularize in process so that you can more easily enforce that certain areas are decoupled from others without having to deal with the headaches of network calls.

Thank you for the explanation. So "in-process microservices" is the final goal. The concept is easy for me to understand. TBH, I am not sure if it is practically very useful (maybe because I don't have monolith experience. I am not against the proposal).

We will discuss this topic tomorrow at the dev meeting. If @matz is positive about the idea, it would be good to have a separate ticket with a clear explanation of the motivation and all the core features needed to implement Im reasonably, i.e., without using fragile hacks like const_missing.

shioyama (Chris Salzberg) wrote in #note-28:
> To be clear, my goal is that Ruby would implement the parts of this problem which are not implementable in gem code, and lets a gem like Im do the rest (similar to Zeitwerk's relationship to `autoload`).

(This is a side note.) I think this is an approach, not a goal. And TBH, I don't think this is the best approach. In my opinion, ideally, language extension-like features such as Zeitwerk and ActiveSupport::Concern should be provided in the core. We are in this situation because Ruby is so flexible to allow such language extensions to be implemented outside, and maybe because it is more lightweight to design and improve a external gem than a core feature, but it's a little disconcerting to say this as if it were an ideal situation.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99229

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:109982] [Ruby master Feature#10320] require into module

[jeremyevans0 (Jeremy Evans)]

Issue #10320 has been updated by jeremyevans0 (Jeremy Evans).


I think that trying to require into a module with code that was not designed for it will break things.  One example is when using absolute constant references (those that start with `::`).  Consider this code:

```ruby
class A; end

class B < BasicObject
  C = ::Object
  D = ::A
end
```

This is an example of where you would generally use absolute constant references, because constant lookup in `BasicObject` will not look up constants in `Object`/top-level.  However, any case where you are using absolute constant references should have this issue.

How would the code above work when loaded into a module?  If absolute constant references are resolved through the module, the access to `::Object` breaks, since that is not defined in the module.  If absolute constant references are not resolved through the module, the access to `::A` breaks, since it would no longer be defined at `Object`/top-level.  Looking in the module first and then `Object`/top-level (or vice-versa) feels ad-hoc, and either approach has corner cases where it breaks.

It looks like `Im` attempts to handle the above case by copying global constants into the module.  I doubt we would want to do that in `load` or `require`.

shioyama (Chris Salzberg) wrote in #note-19:
> > This NilClass definition, even if reassigning global ::NilClass doesn't have any effect on nil though, isn't it?
> Or do you actually define mod::NilClass = NilClass before loading the ActiveSupport files?
> 
> Yes, `mod::NilClass = NilClass` is assigned in the module before passing it to the first `load`, so when loading core extensions ActiveSupport sees `mod::NilClass` and this simply points to `::NilClass`.
> 
> You can confirm it works:
> 
> ```ruby
> nil.blank?
> #  undefined method `blank?' for nil:NilClass (NoMethodError)
> 
> require "im"
> extend Im
> 
> mod = import "active_support"
> #=> <#Im::Import root: active_support>
> 
> ActiveSupport
> # `const_missing': uninitialized constant ActiveSupport (NameError)
> 
> mod::ActiveSupport
> #=> ActiveSupport
> 
> nil.blank?
> #=> true
> ```

To me, this example is a perfect indication of why we shouldn't support this.  This uses `import` to load `ActiveSupport`, so that `ActiveSupport` is not added to top level namespace, but all of the core extensions added by `ActiveSupport` are still active. The namespace isolation is only partial, it is not complete.

There is discussion about how this could allow multiple versions of the same gem versions to work.  How would that work if the gem makes modifications to core classes, as `ActiveSupport` does?  Let's say you are including/prepending a module in the class in both versions, overriding a method, and and calling `super` for default behavior.  Seems like you would get the behavior for both versions, which is unlikely to be desirable.  The situation is worse if a method aliasing approach is used, since running `alias orig_method method; def method; code; orig_method; end` twice would likely result in a method that causes `SystemStackError`.

I'm against `require` accepting a module similar to load, and against making the module wrapping behavior transitive, so that `require` and `load` automatically use the currently wrapping module.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99230

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:110071] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


I think there are some misunderstandings, which are at least partly my fault for not being clearer about my intentions here.

There are two things I presented earlier in this thread:
- changes to Ruby ([here](https://github.com/shioyama/ruby/tree/import_modules)) to make the `load` wrap module transitive in `require`, including native extension code, and some other smaller changes around `name`.
- a gem ([Im](https://github.com/shioyama/im)) which takes advantage of those changes to implement imports

I do not intend to propose here that everything that's in the gem be implemented in Ruby itself. My intention is to motivate changes at the language level necessary to open the door to gems like Im do implement imports in whatever way gem owners see fit.

> It looks like Im attempts to handle the above case by copying global constants into the module. I doubt we would want to do that in `load` or `require`.

Agreed, I had always imagined that this was something that could be done at the gem level, or not at all, depending on needs.

> To me, this example is a perfect indication of why we shouldn't support this. This uses `import` to load `ActiveSupport`, so that `ActiveSupport` is not added to top level namespace, but all of the core extensions added by `ActiveSupport` are still active. The namespace isolation is only partial, it is not complete.

The namespace isolation _as implemented in the gem_ is only partial. The Ruby side should IMO be as close as possible to absolute isolation, so e.g. toplevel references in the wrapped module should resolve to the module top, not absolute top. This way the importer has full control over what constants to make available to the wrapped script.

But this of course poses a problem for basic classes that can be monkeypatched (`String`, `Hash`, etc) I don't know what the solution for that is but it seems to me the problem there is more with monkeypatching as a common practice than it is with imports (potentially) giving access to those classes across multiple imports.

> I'm against `require` accepting a module similar to `load`, and against making the module wrapping behavior transitive, so that `require` and `load` automatically use the currently wrapping module.

To be clear, I'm also against the former. 

The latter (transitivity of the wrap module to `require`) is a dealbreaker though because if we do not have transitivity, then existing code will never be importable without rewriting it. I am not suggesting it has to be `load`, but transitivity _somewhere_ to `require` is going to be necessary in order for any kind of code importing to be practically useful.

The discussion of loading multiple versions of a gem is interesting, but I don't think it's terribly relevant to deciding whether Ruby should support transitivity of wrapping behaviour. What I implemented in the gem, with constant aliases, was one way of dealing with core extensions, and it works (sort of) for loading gems like Rails that monkey-patch core classes. Of course this would cause problems loading multiple versions of a gem, but I'm not suggesting Ruby ever should do this.

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99322

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/

[ruby-core:110098] [Ruby master Feature#10320] require into module

[shioyama (Chris Salzberg)]

Issue #10320 has been updated by shioyama (Chris Salzberg).


At the last Developer Meeting, it was suggested I create a new issue for the topic we've been discussing since [my comment above](https://bugs.ruby-lang.org/issues/10320#note-13) since there are new developments, so I've done that:

https://bugs.ruby-lang.org/issues/19024

----------------------------------------
Feature #10320: require into module
https://bugs.ruby-lang.org/issues/10320#change-99354

* Author: sowieso (So Wieso)
* Status: Open
* Priority: Normal
----------------------------------------
When requiring a library, global namespace always gets polluted, at least with one module name. So when requiring a gem with many dependencies, at least one constant enters global namespace per dependency, which can easily get out of hand (especially when gems are not enclosed in a module).

Would it be possible to extend require (and load, require_relative) to put all content into a custom module and not into global namespace?

Syntax ideas:

~~~ruby
require 'libfile', into: :Lib   # keyword-argument
require 'libfile' in Lib   # with keyword, also defining a module Lib at current binding (unless defined? Lib)
require_qualified 'libfile', :Lib
~~~

This would also make including code into libraries much easier, as it is well scoped.

~~~ruby
module MyGem
  require 'needed' in Need

  def do_something
    Need::important.process!
  end
end
 # library user is never concerned over needed's content
~~~

Some problems to discuss:

* requiring into two different modules means loading the file twice?
* monkeypatching libraries should only affect the module ­→ auto refinements?
* maybe also allow a binding as argument, not only a module?
* privately require, so that required constants and methods are not accessible from the outside of a module (seems to difficult)
* what about $global constants, read them from global scope but copy-write them only to local scope?

Similar issue:
https://bugs.ruby-lang.org/issues/5643



-- 
https://bugs.ruby-lang.org/