[ruby-core:102919] [Ruby master Bug#17730] Ruby on macOS transitively links to ~150 dylibs

[ruby-core:102919] [Ruby master Bug#17730] Ruby on macOS transitively links to ~150 dylibs

[rickmark]

Issue #17730 has been reported by rickmark (Rick Mark).

----------------------------------------
Bug #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730

* Author: rickmark (Rick Mark)
* Status: Open
* Priority: Normal
* ruby -v: 3.0.0
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102921] [Ruby master Bug#17730] Ruby on macOS transitively links to ~150 dylibs

[nobu]

Issue #17730 has been updated by nobu (Nobuyoshi Nakada).


`-lobjc` has been intentionally linked to link startup code with Objective-C support, by the request for extension libraries written in it.


----------------------------------------
Bug #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-90986

* Author: rickmark (Rick Mark)
* Status: Open
* Priority: Normal
* ruby -v: 3.0.0
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102930] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[rickmark]

Issue #17730 has been updated by rickmark (Rick Mark).


Looks like we applied the Foundation -> CoreFoundation patch.

What about the Security framework?  It's the source of the worst dependencies like Metal, CoreML, etc...

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-90996

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102932] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[nobu]

Issue #17730 has been updated by nobu (Nobuyoshi Nakada).


The Security framework is linked only when it is available but CommonRandom is not.
It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
I'm not sure whether other sources are available on these old versions.

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-90997

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102973] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[duerst]

Issue #17730 has been updated by duerst (Martin Dürst).


rickmark (Rick Mark) wrote:

> Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to `m.unicode_normalize(:nfc)`. 


----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91036

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:102974] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[naruse]

Issue #17730 has been updated by naruse (Yui NARUSE).


duerst (Martin Dürst) wrote in #note-6:
> rickmark (Rick Mark) wrote:
> 
> > Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.
> 
> I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to `m.unicode_normalize(:nfc)`.

Why it uses `CFStringNormalize(m, kCFStringNormalizationFormC);` is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.


----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91037

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:102976] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[rickmark]

Issue #17730 has been updated by rickmark (Rick Mark).


nobu (Nobuyoshi Nakada) wrote in #note-5:
> The Security framework is linked only when it is available but CommonRandom is not.
> It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
> I'm not sure whether other sources are available on these old versions.

From compiling recently it looks like `Security.framework` is always linked (see the static linkage results from the text files)

I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)

Will open a new issue for that work

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91039

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102977] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[rickmark]

Issue #17730 has been updated by rickmark (Rick Mark).


naruse (Yui NARUSE) wrote in #note-7:
> duerst (Martin Dürst) wrote in #note-6:
> > rickmark (Rick Mark) wrote:
> > 
> > > Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.
> > 
> > I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to `m.unicode_normalize(:nfc)`.
> 
> Why it uses `CFStringNormalize(m, kCFStringNormalizationFormC);` is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.

Yep - the normalization is called `UTF-8-MAC` on libicu.  This implies we can use other code to do this.  Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B.  This would also allow other platforms to do proper normalization for macOS as well?  I will cut a new ticket for this.

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91040

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>

[ruby-core:102978] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[rickmark]

Issue #17730 has been updated by rickmark (Rick Mark).

File ruby_final.txt added

rickmark (Rick Mark) wrote in #note-8:
> nobu (Nobuyoshi Nakada) wrote in #note-5:
> > The Security framework is linked only when it is available but CommonRandom is not.
> > It is the case that the minimum required macOS version is 10.7(Lion)..10.9(Mavericks).
> > I'm not sure whether other sources are available on these old versions.
> 
> From compiling recently it looks like `Security.framework` is always linked (see the static linkage results from the text files)
> 
> I can rework this patch to do what we would expect (use Security.framework on 10.7-10.9 and then use CoreCrypto for > 10.10)
> 
> Will open a new issue for that work

Nevermind - I just recompiled ruby from head.  Looks like the linking is correct now.  Must have been Foundation brining in Security causing my confusion.  Final linking output attached

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91041

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)
ruby_final.txt (2.24 KB)


-- 
https://bugs.ruby-lang.org/

[ruby-core:102980] [Ruby master Feature#17730] Ruby on macOS transitively links to ~150 dylibs

[naruse]

Issue #17730 has been updated by naruse (Yui NARUSE).


rickmark (Rick Mark) wrote in #note-9:
> naruse (Yui NARUSE) wrote in #note-7:
> > duerst (Martin Dürst) wrote in #note-6:
> > > rickmark (Rick Mark) wrote:
> > > 
> > > > Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.
> > > 
> > > I'm not at all familiar with the details on a Mac, nor with this library or function in particular. But Unicode String Normalization is provided by String#unicode_normalize, which is implemented in pure Ruby. There may be a bit of glue code necessary, but essentially, the above call should be convertible to something close to `m.unicode_normalize(:nfc)`.
> > 
> > Why it uses `CFStringNormalize(m, kCFStringNormalizationFormC);` is because it needs to just the same conversion as HFS+/APFS, which is different from standard NFC.
> 
> Yep - the normalization is called `UTF-8-MAC` on libicu.  This implies we can use other code to do this.  Also since CF is opensource we could just bring in the routine directly and avoid any linkage to core macOS frameworks other then libSystem.B.  This would also allow other platforms to do proper normalization for macOS as well?  I will cut a new ticket for this.

Ruby also has UTF-8-MAC, but HFS+/APFS's one is sometimes updated.
We first used our own encoding table for the usage, but finally we switched to use OS's to ensure the behavior is exactly same as the filesystem.

----------------------------------------
Feature #17730: Ruby on macOS transitively links to ~150 dylibs
https://bugs.ruby-lang.org/issues/17730#change-91042

* Author: rickmark (Rick Mark)
* Status: Closed
* Priority: Normal
----------------------------------------
By using `-framework Security` and `-framework Foundation` Ruby 3 pulls in about 150 dylibs when compiled for macOS

By using CoreCrypto / CoreFoundation I was able to reduce this to ~50.  This greatly reduces Ruby's surface area and dependencies on macOS.  Further CoreFoundation is only used for one call in the entire codebase of `CFStringNormalize(m, kCFStringNormalizationFormC);` - if we can replace this, Ruby could work with only `libSystem` and `libgmp`.

---Files--------------------------------
ruby_deps_after.txt (2.24 KB)
ruby_deps_before.txt (9.81 KB)
0001-Remove-unneeded-dependencies-on-macOS.patch (2.36 KB)
ruby_final.txt (2.24 KB)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>