[ruby-core:72120] [Ruby trunk - Feature #11817] [Open] map.parallel

[ruby-core:72120] [Ruby trunk - Feature #11817] [Open] map.parallel

[eike]

Issue #11817 has been reported by Eike Dierks.

----------------------------------------
Feature #11817: map.parallel
https://bugs.ruby-lang.org/issues/11817

* Author: Eike Dierks
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
I'd like to suggest a new method: #parallel

That method should take care of executing workloads in parallel across multiple cores.

I believe that #map can sometimes be distributed to many simultaneous threads.

So we would spread out the map blocks into parallel processing,
and then collect the results in order.

I believe this would need some new infrastructure like apple did with great_central,
so I'm talking along these lines.
(Aka we only want to open up as many threads as there are cores)

This would not help much with a simple map (because we need to collect and serialize the results in the end)
but it could be cool for more complex mappings.

Maybe this should have been a rails topic first,
because there it might make a lot of sense to render partials in parallel,
this could really speed up rails response rendering ;-)

But I believe this is a topic for ruby in general.
Given that we now go to really many cores,
it should be at the heart of ruby to use that.

For a start we could just declare #parallel as -> return self.
and than improve on it.

A naive implementation would just open a new thread for every parallel map block,
but that would not work well -- it actually would be very slow.

So we actually need a thread management,
along the lines of great central.

I believe this should be built into the very core libs of the ruby language.

I know this is hard stuff, but it needs to be done.
Let's go parallel from the heart of the language.

---


Please let us discuss if adding the #parallel method would be a good idea.
Let's come up with some implementations to find the problems.

I'm explicitly referring to map#parallel here.
In rails that might be cache.parallel (if you followed)

for some math tasks it might also be
   parallel do {}



































-- 
https://bugs.ruby-lang.org/

[ruby-core:72131] [Ruby trunk - Feature #11817] map.parallel

[matz]

Issue #11817 has been updated by Yukihiro Matsumoto.


Show me your imaginary code example, so we can understand your concrete intention.
There's lot of issues around concurrency in Ruby. Simply adding `map.parallel` may not solve the problem.
But we are interested in your idea.

Matz.


----------------------------------------
Feature #11817: map.parallel
https://bugs.ruby-lang.org/issues/11817#change-55542

* Author: Eike Dierks
* Status: Open
* Priority: Normal
* Assignee: 
----------------------------------------
I'd like to suggest a new method: #parallel

That method should take care of executing workloads in parallel across multiple cores.

I believe that #map can sometimes be distributed to many simultaneous threads.

So we would spread out the map blocks into parallel processing,
and then collect the results in order.

I believe this would need some new infrastructure like apple did with great_central,
so I'm talking along these lines.
(Aka we only want to open up as many threads as there are cores)

This would not help much with a simple map (because we need to collect and serialize the results in the end)
but it could be cool for more complex mappings.

Maybe this should have been a rails topic first,
because there it might make a lot of sense to render partials in parallel,
this could really speed up rails response rendering ;-)

But I believe this is a topic for ruby in general.
Given that we now go to really many cores,
it should be at the heart of ruby to use that.

For a start we could just declare #parallel as -> return self.
and than improve on it.

A naive implementation would just open a new thread for every parallel map block,
but that would not work well -- it actually would be very slow.

So we actually need a thread management,
along the lines of great central.

I believe this should be built into the very core libs of the ruby language.

I know this is hard stuff, but it needs to be done.
Let's go parallel from the heart of the language.

---


Please let us discuss if adding the #parallel method would be a good idea.
Let's come up with some implementations to find the problems.

I'm explicitly referring to map#parallel here.
In rails that might be cache.parallel (if you followed)

for some math tasks it might also be
   parallel do {}



































-- 
https://bugs.ruby-lang.org/