[ruby-core:92848] [Ruby trunk Feature#15797] Use realpath(3) instead of custom realpath implementation if available

[merch-redmine@jeremyevans.net]

Issue #15797 has been updated by jeremyevans0 (Jeremy Evans).


I have added a pull request for this feature (https://github.com/ruby/ruby/pull/2205), and made the necessary changes so it passes Travis and AppVeyor.  This changes related to fixing encoding issues and working around issues in glibc realpath(3) returning ENOTDIR and ENOENT in some cases that the current recursive realpath implementation handles correctly (falling back to the current implementation in those cases if realpath(3) returns NULL).

From some research, it looks like realpath(3) on Mac OS 10.5 and below does not handle a NULL second argument.  Does Ruby still support Mac OS X versions that old (10.5 was released October 2007)?  If so, I can update configure.ac to check for realpath(3) supporting a NULL second argument if the function itself is supported.

Can anyone run the benchmark in the pull request on Linux and Mac OS X (and possibly other operating systems) and post the results?

----------------------------------------
Feature #15797: Use realpath(3) instead of custom realpath implementation if available
https://bugs.ruby-lang.org/issues/15797#change-78229

* Author: jeremyevans0 (Jeremy Evans)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
----------------------------------------
One reason to do this is simplicity, as this approach is ~30 lines of
code instead of ~200.

Performance wise, this performs 25%-115% better, using the following
benchmark on OpenBSD 6.5:

```ruby
require 'benchmark'

f = File
pwd = Dir.pwd
Dir.mkdir('b') unless f.directory?('b')
f.write('b/a', '') unless f.file?('b/a')

args = [
  ["b/a", nil],
  ["#{pwd}/b/a", nil],
  ['a', 'b'],
  ["#{pwd}/b/a", 'b'],
  ["b/a", pwd]
]

args.each do |path, base|
  print "File.realpath(#{path.inspect}, #{base.inspect}): ".ljust(50)
  puts Benchmark.measure{100000.times{f.realpath(path, base)}}
end
```

Before:

```
File.realpath("b/a", nil):                          4.330000   2.990000   7.320000 (  7.316244)
File.realpath("/home/testr/ruby/b/a", nil):         3.560000   2.680000   6.240000 (  6.240951)
File.realpath("a", "b"):                            4.370000   3.080000   7.450000 (  7.452511)
File.realpath("/home/testr/ruby/b/a", "b"):         3.730000   2.640000   6.370000 (  6.371979)
File.realpath("b/a", "/home/testr/ruby"):           3.590000   2.630000   6.220000 (  6.226824)
```

After:

```
File.realpath("b/a", nil):                          1.370000   2.030000   3.400000 (  3.400775)
File.realpath("/home/testr/ruby/b/a", nil):         1.260000   2.770000   4.030000 (  4.024957)
File.realpath("a", "b"):                            2.090000   1.990000   4.080000 (  4.080284)
File.realpath("/home/testr/ruby/b/a", "b"):         1.400000   2.620000   4.020000 (  4.015505)
File.realpath("b/a", "/home/testr/ruby"):           2.150000   2.760000   4.910000 (  4.910634)
```

If someone could benchmark before/after with this patch on Linux and/or MacOS X,
and post the results here, I would appreciate it.

My personal reason for wanting this is that the custom realpath
implementation does not work with OpenBSD's unveil(2) system call,
which limits access to the file system, allowing for security
similar to chroot(2), without most of the downsides.

This change passes all tests except for one assertion related to
taintedness.  Previously, if either argument to `File.realpath` is an
absolute path, then the returned value is considered not tainted.
However, I believe that behavior to be incorrect, because if there is
a symlink anywhere in the path, the returned value can contain a
section that was taken from the file system (unreliable source) that
was not marked as untainted. Example:

```ruby
Dir.mkdir('b') unless File.directory?('b')
File.write('b/a', '') unless File.file?('b/a')
File.symlink('b', 'c') unless File.symlink?('c')
path = File.realpath('c/a'.untaint, Dir.pwd.untaint)
path # "/home/testr/ruby/b/a"
path.tainted? # should be true, as 'b' comes from file system
```

I believe it is safer to always mark the output of realpath as tainted
to prevent this issue, which is what this commit does.

---Files--------------------------------
use-native-realpath.patch (6.31 KB)
use-native-realpath-v2.patch (4.64 KB)
use-native-realpath-v3.patch (5.18 KB)
use-native-realpath-v4.patch (6.34 KB)


-- 
https://bugs.ruby-lang.org/