[ruby-core:103197] [Ruby master Feature#5446] at_fork callback API

[ruby-core:103197] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


Could this `at_exit` proposal be considered again? I understand and agree that in theory libraries needed to protect against forks should expose a `after_fork` method to be called after forking, and that it's the users's responsibility to hook the two properly.

But in practice many libraries prefer to check `Process.pid`, and often end up doing so in tight loops. And they do it so much that it end up being significant on performance profiles in real production environments.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91273

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103198] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


> Could this `at_exit` proposal be considered again?

Sorry, I meant `at_fork`

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91274

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103199] Re: [Ruby master Feature#5446] at_fork callback API

[Eric Wong]

jean.boussier@gmail.com wrote:
> Feature #5446: at_fork callback API
> https://bugs.ruby-lang.org/issues/5446#change-91273
> 
> But in practice many libraries prefer to check `Process.pid`,
> and often end up doing so in tight loops. And they do it so
> much that it end up being significant on performance profiles
> in real production environments.

Some things have changed in over the years:

* getpid() no longer cached by glibc, so it's always a syscall, now

* syscalls are more expensive due to CPU vulnerability mitigations

* Ractor becoming usable/available in Ruby 3+

* system/popen/backtick use vfork() since Ruby 2.2

* JIT exists:

Is there a way to expose Ruby methods/procs as C function pointers with JIT?

If so, Fiddle may be usable to call pthread_atfork(3) and have
it use JIT-exposed C function pointers.

That would be generally useful for interacting with other C libraries,
not just for pthread_atfork.

[ruby-core:103202] [Ruby master Feature#5446] at_fork callback API

[eregontp]

Issue #5446 has been updated by Eregon (Benoit Daloze).


> Is there a way to expose Ruby methods/procs as C function pointers with JIT?

@normalperson That feels very hacky to me.
JIT'ing a function which is only called once seems suboptimal (and difficult to do before it's even called once).
Also some Ruby state likely needs to be restored before running any `after_fork` hook.

@byroot I agree, we should at least have `after_fork` (or `at_fork(when, &block)`).

Honestly I don't understand why we don't have this yet, it's obviously needed and every forking webserver out there ends up having its own hook.

For the use-case of purposefully keeping a database connection (which seems extremely rare), maybe one can require the database library with a different path to be explicit that playing with database fd sharing fire is wanted? (or some other mechanism so the `at_fork` hook is not installed in that case).

For the vast majority of fork uses in practice it would be much safer if database connections and other fds which are unsafe to keep open across forks were automatically closed.

And BTW the current workaround of checking pid is slower also on platforms not supporting `fork`, so it is inefficient for all platforms and webservers.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91277

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103203] [Ruby master Feature#5446] at_fork callback API

[eregontp]

Issue #5446 has been updated by Eregon (Benoit Daloze).


Eregon (Benoit Daloze) wrote in #note-38:
> require the database library with a different path

Or by having some logic in e.g. rack to handle that. I'm sure we can find something.
But the hook needs to be in core, otherwise we'd need to monkey-patch Kernel#fork, and that wouldn't work for `rb_f_fork`.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91278

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103205] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


> I agree, we should at least have `after_fork`

Yes, that's what's needed 99% of the time.

[We implemented it in Active Support a few years back](https://github.com/rails/rails/blob/80bd07a61a96a77dcc8161621ec5275f2d1e9486/activesupport/lib/active_support/fork_tracker.rb), but many more libraries would need it, and it doesn't feel clean, and I feel bad doing the same thing in other libraries.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91280

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103206] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


To clarify, I don't think we even need native like `pthread_atfork`, as as far as I know it's not safe for extensions to use `fork(2)` unless of course they immediately `exec()`.

So what I'm asking for is simply for `Process.fork` to call a `after_fork` callback in the child. Maybe `Process.after_fork = Proc`? 

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91281

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103207] [Ruby master Feature#5446] at_fork callback API

[daniel]

Issue #5446 has been updated by Dan0042 (Daniel DeLorme).


> For the vast majority of fork uses in practice it would be much safer if database connections and other fds which are unsafe to keep open across forks were automatically closed.

Indeed, and it's possible to do that with `IO#close_on_exec`, which seems to me like the appropriate solution in 99% of cases where an after_fork hook is requested. So while I agree in principle that after_fork would be useful, it would be nice to see a use-case that just _cannot_ be solved with `IO#close_on_exec`

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91282

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103208] Re: [Ruby master Feature#5446] at_fork callback API

[Eric Wong]

eregontp@gmail.com wrote:
> > Is there a way to expose Ruby methods/procs as C function pointers with JIT?
> 
> @normalperson That feels very hacky to me.

fork() seems hacky now that Ractor exists, and
Process.spawn/popen/system use vfork (along with cloexec being
the default)

> JIT'ing a function which is only called once seems suboptimal
> (and difficult to do before it's even called once).
> Also some Ruby state likely needs to be restored before
> running any `after_fork` hook.

Maybe not "JIT", but just-ahead-of-time or on-demand-compiled.
Something like "Proc#compile_to_c" returning an Integer pointer.

It would be generally useful for Fiddle to pass callbacks
implemented in Ruby around, but I agree tricky to get right
because Ruby state could be indeterminate when the C function is
called...

> @byroot I agree, we should at least have `after_fork` (or `at_fork(when, &block)`).

For projects where I'm stuck having to fork, I find "prepare"
(before_fork) just as useful.  I've only used "parent"
(after fork), once, however.

> Honestly I don't understand why we don't have this yet, it's
> obviously needed and every forking webserver out there ends up
> having its own hook.

With Ractor, perhaps fork() shouldn't be encouraged/promoted, anymore.

> And BTW the current workaround of checking pid is slower also
> on platforms not supporting `fork`, so it is inefficient for
> all platforms and webservers.

Agreed.

[ruby-core:103209] Re: [Ruby master Feature#5446] at_fork callback API

[Eric Wong]

Eric Wong wrote:
> fork() seems hacky now that Ractor exists, and
> Process.spawn/popen/system use vfork (along with cloexec being
> the default)

<snip>

> With Ractor, perhaps fork() shouldn't be encouraged/promoted, anymore.

Expanding on that:  fork() is a middle-of-the-road approach;
like roadkill, it gets run over from both sides.

Ractor uses less memory, starts faster, and makes sharing data easier.

Process.spawn can give the ultimate in isolation and data safety
(at the expense of startup performance).

fork() has high/unpredictable memory use and performance
dependent on CoW (which is still tricky to predict and control).
It also has difficult and expensive data sharing; along with
potential for inadvertant data leaks.


By the time users adopt a new Ruby API for atfork, I think
they'd be better served by moving to Ractor, instead.

[ruby-core:103210] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


> it would be nice to see a use-case that just cannot be solved with IO#close_on_exec

The need is not limited to IOs. If you have say some kind of buffered event queue ([example](https://github.com/Shopify/statsd-instrument/pull/280)), you need to clear it after fork otherwise you will send some events twice.

> fork() seems hacky now that Ractor exists

Seems kind of out of topic to discuss the merits of threads vs process, but even then I wouldn't say so. First Ractors still aren't quite production ready, but even if they were, standalone processes still provide extra isolation that is in some ways superior to threads/ractors. When unicorn timeouts a workers, since it's a distinct process, you know the replacement will be spawned clean. If your workers are thread/ractors you either have to be certain you won't need to kill them, or be certain that your whole software stack will handle it cleanly. And from experience, most code out there doesn't (no blame here, it's incredibly hard to do).

Of course you can always consider that a service that get stuck or slow and need to be killed is a bug in itself, but this kind of resiliency as a last line of defence is appreciable.



----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91284

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103212] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


> it would be nice to see a use-case that just cannot be solved with IO#close_on_exec

Another use case is if you have a dispatcher thread, it will need to be restarted after fork.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91286

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103235] [Ruby master Feature#5446] at_fork callback API

[daniel]

Issue #5446 has been updated by Dan0042 (Daniel DeLorme).


> > For the vast majority of fork uses in practice it would be much safer if database connections and other fds which are unsafe to keep open across forks were automatically closed.
> 
> Indeed, and it's possible to do that with `IO#close_on_exec`,

I apologize, somehow I was thinking that `fork` belongs to the `exec` family of functions and so `IO#close_on_exec` applied to it, but that's obviously not the case. Since ruby doesn't have something like `IO#close_on_fork` then yeah having an `after_fork` hook is very necessary.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91316

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103254] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


I'm thinking a simpler alternative would be to make `Process.fork` the unique forking method: https://github.com/ruby/ruby/pull/4361

This would make decorating `Process.fork` trivial, effectively offering an easy way to register a callback on either before or after fork.


----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91337

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103297] [Ruby master Feature#5446] at_fork callback API

[ivo.anjo]

Issue #5446 has been updated by ivoanjo (Ivo Anjo).


+1 we have this issue at Datadog as well. Our profiler keeps a background thread that takes samples and reports back performance information, and we'd like to support keeping the profiler enabled even when processes fork. (Our yet-another-at_fork is https://github.com/DataDog/dd-trace-rb/blob/feature/profiling/lib/ddtrace/profiling/ext/forking.rb ).

@byroot's solution I think would be a great improvement, as at least we'd be able to just prepend our needed behavior.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91383

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103393] [Ruby master Feature#5446] at_fork callback API

[mame]

Issue #5446 has been updated by mame (Yusuke Endoh).


Can anyone summarize the latest proposal? I cannot understand what is needed precisely. I'm afraid if some people see only the name "at_fork", and do not share the context.

What event should be hooked? Only a call to Kernel#fork and Process.fork? Also Process.daemon? Kernel#system, #spawn and IO#popen? Or fork(2) system call? Some talk about the redefinition of `Kernel#fork`, but the others do about `pthread_atfork`. I guess they should be distinguished.

I understand the motivation. Multiple gems implement their own fork hook system, which is not good. But are the features the same? Did anyone study them? I guess we need to provide the (reasonably) greatest common feature set.

The discussion is too long (for me) to follow, so it may be helpful to create another ticket with a brief description of use-case study and detailed proposal.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91487

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103397] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


@mame I'll do it.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91491

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103399] [Ruby master Feature#5446] at_fork callback API

[mame]

Issue #5446 has been updated by mame (Yusuke Endoh).


byroot (Jean Boussier) wrote in #note-50:
> @mame I'll do it.

Thanks!

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91494

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103401] [Ruby master Feature#5446] at_fork callback API

[jean.boussier]

Issue #5446 has been updated by byroot (Jean Boussier).


@mame I created https://bugs.ruby-lang.org/issues/17795, let me know if it makes sense to you, in which case I'll update my developer meeting request.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91495

* Author: normalperson (Eric Wong)
* Status: Assigned
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/

[ruby-core:103426] [Ruby master Feature#5446] at_fork callback API

[mame]

Issue #5446 has been updated by mame (Yusuke Endoh).

Status changed from Assigned to Closed

byroot (Jean Boussier) wrote in #note-52:
> @mame I created https://bugs.ruby-lang.org/issues/17795, let me know if it makes sense to you, in which case I'll update my developer meeting request.

Thank you, it is very clear!

I'm closing this ticket since its goal was drifted over the years. If anyone is not satisfied, let me know.

----------------------------------------
Feature #5446: at_fork callback API
https://bugs.ruby-lang.org/issues/5446#change-91519

* Author: normalperson (Eric Wong)
* Status: Closed
* Priority: Normal
* Assignee: kosaki (Motohiro KOSAKI)
----------------------------------------
It would be good if Ruby provides an API for registering fork() handlers.

This allows libraries to automatically and agnostically reinitialize resources
such as open IO objects in child processes whenever fork() is called by a user
application.  Use of this API by library authors will reduce false/improper
sharing of objects across processes when interacting with other
libraries/applications that may fork.

This Ruby API should function similarly to pthread_atfork() which allows
(at least) three different callbacks to be registered:

1) prepare - called before fork() in the original process
2) parent - called after fork() in the original process
3) child - called after fork() in the child process

It should be possible to register multiple callbacks for each action
(like at_exit and pthread_atfork(3)).

These callbacks should be called whenever fork() is used:

- Kernel#fork
- IO.popen
- ``
- Kernel#system

... And any other APIs I've forgotten about

I also want to consider handlers that only need to be called for plain
fork() use (without immediate exec() afterwards, like with `` and system()).

Ruby already has the internal support for most of this this to manage mutexes,
Thread structures, and RNG seed.  Currently, no external API is exposed.  I can
prepare a patch if an API is decided upon.




-- 
https://bugs.ruby-lang.org/