[ruby-core:22893] [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[ruby-core:22893] [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[David Martin]

Feature #1291: O_CLOEXEC flag missing for Kernel::open
http://redmine.ruby-lang.org/issues/show/1291

Author: David Martin
Status: Open, Priority: Normal
Category: core

Linux has a the most useful O_CLOEXEC flag for open() that sets the CLOEXEC flag on the new file descriptor.

You can currently set the CLOEXEC flag on an open file descriptor using IO::fcntl(), but note that this does *not* work in a multithreaded program:  If one thread does open/fcntl while another does an exec, there is a race condition that could produce a file descriptor leak.  The only safe way to open a file with the CLOEXEC flag set in general (as far as I know) is to use the O_CLOEXEC flag to open().


----------------------------------------
http://redmine.ruby-lang.org

[ruby-core:22896] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Nobuyoshi Nakada]

Hi,

At Sun, 15 Mar 2009 09:14:24 +0900,
David Martin wrote in [ruby-core:22893]:
> Linux has a the most useful O_CLOEXEC flag for open() that
> sets the CLOEXEC flag on the new file descriptor.
> 
> You can currently set the CLOEXEC flag on an open file
> descriptor using IO::fcntl(), but note that this does *not*
> work in a multithreaded program: If one thread does
> open/fcntl while another does an exec, there is a race
> condition that could produce a file descriptor leak.  The
> only safe way to open a file with the CLOEXEC flag set in
> general (as far as I know) is to use the O_CLOEXEC flag to
> open().

Is it Linux only?

-- 
Nobu Nakada

[ruby-core:22899] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Tanaka Akira]

In article <49bc7c3c.05886e0a.09ae.ffffde11@mx.google.com>,
  Nobuyoshi Nakada <nobu@ruby-lang.org> writes:

> Is it Linux only?

Yes.

I think O_CLOEXEC should be used internally if Ruby sets
close-on-exec flag for file descriptors, as Perl.
(See $SYSTEM_FD_MAX in perlvar.)

I guess Ruby may do it in future to avoid the race
condition.  The problem may be more realistic with MVM.

However it changes system and exec incompatibly (on Unix).
If it is changed and someone really want to inherit a file
descriptor, system("command") need to be changed to
system("command", fd=>fd).

IO.popen and spawn doesn't affected because they close
unrelated file descriptors by default.
-- 
Tanaka Akira

[ruby-core:29038] [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Motohiro KOSAKI]

Issue #1291 has been updated by Motohiro KOSAKI.

File 0001-O_CLOEXEC.patch added

Sure. This is linux specific feature. and I attached the proposal patch.

test way:

test.rb
--------------------------
open("foo", File::CREAT|File::RDWR|File::CLOEXEC, 0644)

strace -e open ruby test.rb
-------------------------------
(snip)
open("./test.rb", O_RDONLY)             = 3
open("foo", O_RDWR|O_CREAT|O_CLOEXEC, 0644) = 3


btw, glibc fopen(3) has "e" mode extention. It mean fopen("foo", "r+e") is equivalent
open("foo", O_RDWR|O_CLOEXEC). but I don't change mode spec of File::open, because
I don't think this is enough widly used nor enough widly known from people.

thanks.

----------------------------------------
http://redmine.ruby-lang.org/issues/show/1291

----------------------------------------
http://redmine.ruby-lang.org

[ruby-core:29039] [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Motohiro KOSAKI]

Issue #1291 has been updated by Motohiro KOSAKI.

File 0001-O_CLOEXEC.patch added

Grr, I forgot to fix commnet. fixed.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1291

----------------------------------------
http://redmine.ruby-lang.org

[ruby-core:29040] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[stygian23]

[-- Attachment #1: Type: TEXT/plain, Size: 509 bytes --]

How do I unsubscribe to this, as its currently flooding my work e-mail and I don't have the time to follow my intellectual curiosity.

In a message dated 03/26/10 09:48:43 Eastern Standard Time, redmine@ruby-lang.org writes:
Issue #1291 has been updated by Motohiro KOSAKI. 

File 0001-O_CLOEXEC.patch added 

Grr, I forgot to fix commnet. fixed. 
---------------------------------------- 
http://redmine.ruby-lang.org/issues/show/1291 

---------------------------------------- 
http://redmine.ruby-lang.org 

[-- Attachment #2: Type: TEXT/html, Size: 1401 bytes --]

<HTML><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18876"></HEAD>
<BODY bottomMargin=7 leftMargin=7 rightMargin=7 topMargin=7>
<DIV><FONT size=2 face=Arial>How do I unsubscribe to this, as its currently flooding my work e-mail and I don't have the time to follow my intellectual curiosity.</FONT></DIV>
<DIV>&nbsp;</DIV>
<DIV>In a message dated 03/26/10 09:48:43 Eastern Standard Time, redmine@ruby-lang.org writes:</DIV>
<BLOCKQUOTE style="BORDER-LEFT: blue 2px solid; PADDING-LEFT: 5px; MARGIN-LEFT: 5px">
<DIV>
<STYLE type=text/css>
.aolmailheader          {font-size:8pt; color:black; font-family:Arial}
a.aolmailheader:link    {color:blue; text-decoration:underline; font-weight:normal}
a.aolmailheader:visited {color:magenta; text-decoration:underline; font-weight:normal}
a.aolmailheader:active  {color:blue; text-decoration:underline; font-weight:normal}
a.aolmailheader:hover   {color:blue; text-decoration:underline; font-weight:normal}
</STYLE>
Issue #1291 has been updated by Motohiro KOSAKI. <BR><BR>File 0001-O_CLOEXEC.patch added <BR><BR>Grr, I forgot to fix commnet. fixed. <BR>---------------------------------------- <BR>http://redmine.ruby-lang.org/issues/show/1291 <BR><BR>---------------------------------------- <BR>http://redmine.ruby-lang.org <BR><BR></DIV></BLOCKQUOTE>
<DIV>&nbsp;</DIV></BODY></HTML>

[ruby-core:29043] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Nikolai Weibull]

On Fri, Mar 26, 2010 at 16:02, stygian23 <stygian23@aol.com> wrote:
> How do I unsubscribe to this, as its currently flooding my work e-mail and I
> don't have the time to follow my intellectual curiosity.

Nor time to Google [1] for an answer, it seems.

[1] http://www.google.se/search?q=ruby-core+unsubscribe

[ruby-core:29059] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Nobuyoshi Nakada]

Hi,

At Fri, 26 Mar 2010 23:46:22 +0900,
Motohiro KOSAKI wrote in [ruby-core:29039]:
> File 0001-O_CLOEXEC.patch added

I'm not against it, although I hope it should be automatic in
the future as mentioned in [ruby-core:22899].

Also, IMHO, all those constants should be defined on all
platforms for portability, like as File::BINARY.

-- 
Nobu Nakada

[ruby-core:29062] [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Motohiro KOSAKI]

Issue #1291 has been updated by Motohiro KOSAKI.


> I'm not against it, although I hope it should be automatic in
> the future as mentioned in [ruby-core:22899].

Umm, sorry, I haven't catch your mention. I think IO::close_on_exec
can only be used for already opened file. I think they have different 
semantics.

If you mean introducing new special variable, I'm not againt it.

> Also, IMHO, all those constants should be defined on all
> platforms for portability, like as File::BINARY.

No. I dislike it. open(O_CLOEXEC) and fcntl(FD_CLOEXEC) are differenct
security meaning. To provide O_CLOEXEC emulation logic mean to make 
security issue. Please remember why O_CLOEXEC was introduced although 
fcntl(FD_CLOEXEC) was already existed.

 -> see http://udrepper.livejournal.com/20407.html

note: "set close-on-exec by default" and "to emulate O_CLOEXEC" are
perfectly different topic. I only againt latter.

- kosaki
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1291

----------------------------------------
http://redmine.ruby-lang.org

[ruby-core:29065] Re: [Feature #1291] O_CLOEXEC flag missing for Kernel::open

[Tanaka Akira]

2010/3/27 Motohiro KOSAKI <redmine@ruby-lang.org>:

> No. I dislike it. open(O_CLOEXEC) and fcntl(FD_CLOEXEC) are differenct
> security meaning. To provide O_CLOEXEC emulation logic mean to make
> security issue. Please remember why O_CLOEXEC was introduced although
> fcntl(FD_CLOEXEC) was already existed.
>
>  -> see http://udrepper.livejournal.com/20407.html

My intent behind [ruby-core:22899] is to fix the race shown here.

O_CLOEXEC and friends (F_DUPFD_CLOEXEC, etc.) are good tool to avoid the
race condition.
They is defined in POSIX now.

However they are not popular.
I don't know platforms with O_CLOEXEC other than Linux.
But the race condition exists not only on Linux.
I want to fix the race on all platforms.

Defining File::CLOEXEC as O_CLOEXEC is not enough to fix the race.
Applications should be modified to use it.
Since O_CLOEXEC is not portable, it should be used conditionally as:

  f = open(filename, File::WRONLY|(defined?(File::CLOEXEC)?File::CLOEXEC:0))
  f.close_on_exec = true

It is too complicated to believe usual programmers do it for all open call.

So, my idea is modifying Ruby (incompatibly) to set CLOEXEC flag by
* using O_CLOEXEC for all open() on Linux, and
* using FD_CLOEXEC immediately after open() on non-Linux.

Actually this minimize the race instead of fix on non-Linux.
But I don't have practical idea better than this.

Since O_CLOEXEC is standardized by POSIX, I guess this idea can fix
the race in many platforms in future.

I think POSIX cannot change open() to set CLOEXEC by default because
compatibility.

But I think Ruby can accept this incompatibility.

Apart from my idea, defining File::CLOEXEC as 0 on non-Linux simplify the code:

  f = open(filename, File::WRONLY|File::CLOEXEC)
  f.close_on_exec = true

Defining File::CLOEXEC as some number to emulate O_CLOEXEC simplify more:

  f = open(filename, File::WRONLY|File::CLOEXEC)

So I don't against O_CLOEXEC emulation.
It makes easier to minimize the race.

I still don't believe usual programmers uses it, though.
-- 
Tanaka Akira