* [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references
@ 2012-04-17 8:10 headius (Charles Nutter)
2012-04-19 19:51 ` [ruby-core:44470] [ruby-trunk - Feature #6309][Feedback] " mame (Yusuke Endoh)
` (13 more replies)
0 siblings, 14 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2012-04-17 8:10 UTC (permalink / raw
To: ruby-core
Issue #6309 has been reported by headius (Charles Nutter).
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309
Author: headius (Charles Nutter)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:44470] [ruby-trunk - Feature #6309][Feedback] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
@ 2012-04-19 19:51 ` mame (Yusuke Endoh)
2012-04-28 5:47 ` [ruby-core:44719] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
` (12 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mame (Yusuke Endoh) @ 2012-04-19 19:51 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by mame (Yusuke Endoh).
Status changed from Open to Feedback
There is no maintainer for weakref, so please create a patch yourself!
We may import it if matz accepts.
BTW, the name "reference queue" is very bad, I think.
--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-26025
Author: headius (Charles Nutter)
Status: Feedback
Priority: Normal
Assignee:
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:44719] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
2012-04-19 19:51 ` [ruby-core:44470] [ruby-trunk - Feature #6309][Feedback] " mame (Yusuke Endoh)
@ 2012-04-28 5:47 ` headius (Charles Nutter)
2012-05-03 2:29 ` [ruby-core:44826] " mame (Yusuke Endoh)
` (11 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2012-04-28 5:47 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Ok, fair enough.
Here is a *very primitive* modification of the current weakref.rb to support a reference queue. I need to stress that I don't think this is the best way to implement it; a hook into the GC cycle that inserts weakrefs into a purpose-built reference queue would be better than using finalizers in this way. But the API would largely work the same.
Patch: https://gist.github.com/2516338
Example usage: https://gist.github.com/2516355
This works mostly like I expect a reference queue to work, but there are many inefficiencies here:
* Polling the reference queue needs to be as close to free as possible. The current Queue implementation raises an exception when empty, which is very far from being free.
* A Ruby-level finalizer is much more expensive than a purpose-built native GC hook would be.
* A Ruby-based Queue is much more expensive than a purpose-built reference queue would be.
I know that in past discussions about improving weakref support in Ruby there were C-level patches to add all the features I'm looking for, and I'll try to dig up those discussions and patches. But hopefully this illustrates what I'm looking for in a primitive way.
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-26280
Author: headius (Charles Nutter)
Status: Feedback
Priority: Normal
Assignee:
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:44826] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
2012-04-19 19:51 ` [ruby-core:44470] [ruby-trunk - Feature #6309][Feedback] " mame (Yusuke Endoh)
2012-04-28 5:47 ` [ruby-core:44719] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
@ 2012-05-03 2:29 ` mame (Yusuke Endoh)
2012-05-03 2:50 ` [ruby-core:44827] [ruby-trunk - Feature #6309][Assigned] " mame (Yusuke Endoh)
` (10 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mame (Yusuke Endoh) @ 2012-05-03 2:29 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by mame (Yusuke Endoh).
Ah, I knew what you are proposing by seeing Javadoc:
http://docs.oracle.com/javase/1.4.2/docs/api/java/lang/ref/ReferenceQueue.html
http://docs.oracle.com/javase/6/docs/api/java/lang/ref/WeakReference.html#WeakReference(T, java.lang.ref.ReferenceQueue)
I don't know the (real-world) use case of the feature, though.
Anyway, I mean I'd like you to create a patch written *in C*.
If there is a patch that we can review and import "as is",
I will be happy to assign this ticket to some core committers,
such as ko1 and kosaki.
--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-26401
Author: headius (Charles Nutter)
Status: Feedback
Priority: Normal
Assignee:
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:44827] [ruby-trunk - Feature #6309][Assigned] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (2 preceding siblings ...)
2012-05-03 2:29 ` [ruby-core:44826] " mame (Yusuke Endoh)
@ 2012-05-03 2:50 ` mame (Yusuke Endoh)
2012-05-03 20:59 ` [ruby-core:44857] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
` (9 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mame (Yusuke Endoh) @ 2012-05-03 2:50 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by mame (Yusuke Endoh).
Status changed from Feedback to Assigned
Assignee set to matz (Yukihiro Matsumoto)
On second thought, the proposal should first get an approval from matz. Sorry. Assigning this to him.
Still, it would be helpful to show a concrete use case, I think.
--
Yusuke Endoh <mame@tsg.ne.jp>
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-26402
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:44857] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (3 preceding siblings ...)
2012-05-03 2:50 ` [ruby-core:44827] [ruby-trunk - Feature #6309][Assigned] " mame (Yusuke Endoh)
@ 2012-05-03 20:59 ` headius (Charles Nutter)
2012-11-16 16:26 ` [ruby-core:49436] " headius (Charles Nutter)
` (8 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2012-05-03 20:59 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
I linked to a concrete use case in the original report...an implementation of a "weak ID map" entirely in Ruby without scanning for dead references: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
It is not possible to implement weak data structures efficiently without a reference queue, since you would be forced to periodically do an O(N) scan for dead references to clean them out.
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-26432
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:49436] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (4 preceding siblings ...)
2012-05-03 20:59 ` [ruby-core:44857] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
@ 2012-11-16 16:26 ` headius (Charles Nutter)
2012-11-24 1:50 ` [ruby-core:49957] " mame (Yusuke Endoh)
` (7 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2012-11-16 16:26 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Seven months and no activity. Can we get a reference queue in Ruby 2.0 please? I believe it could be added to weakref.rb using 2.0's WeakHash, or built atop the C code that implements WeakHash (since it contains most of a reference queue implementation already).
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-32973
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version:
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:49957] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (5 preceding siblings ...)
2012-11-16 16:26 ` [ruby-core:49436] " headius (Charles Nutter)
@ 2012-11-24 1:50 ` mame (Yusuke Endoh)
2013-03-15 17:28 ` [ruby-core:53451] " headius (Charles Nutter)
` (6 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: mame (Yusuke Endoh) @ 2012-11-24 1:50 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by mame (Yusuke Endoh).
Target version set to next minor
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-33723
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: next minor
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:53451] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (6 preceding siblings ...)
2012-11-24 1:50 ` [ruby-core:49957] " mame (Yusuke Endoh)
@ 2013-03-15 17:28 ` headius (Charles Nutter)
2013-09-27 11:19 ` [ruby-core:57438] " headius (Charles Nutter)
` (5 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-03-15 17:28 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
I request a ruling by matz about adding a ReferenceQueue to weakref.rb.
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-37635
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: next minor
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:57438] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (7 preceding siblings ...)
2013-03-15 17:28 ` [ruby-core:53451] " headius (Charles Nutter)
@ 2013-09-27 11:19 ` headius (Charles Nutter)
2013-09-30 13:21 ` [ruby-core:57491] " headius (Charles Nutter)
` (4 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-09-27 11:19 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
I again request approval from matz to add this feature :-) Can we do it for 2.1, please?
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-42044
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: next minor
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:57491] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (8 preceding siblings ...)
2013-09-27 11:19 ` [ruby-core:57438] " headius (Charles Nutter)
@ 2013-09-30 13:21 ` headius (Charles Nutter)
2013-10-01 1:23 ` [ruby-core:57514] " headius (Charles Nutter)
` (3 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-09-30 13:21 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Target version changed from next minor to current: 2.1.0
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-42104
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: current: 2.1.0
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:57514] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (9 preceding siblings ...)
2013-09-30 13:21 ` [ruby-core:57491] " headius (Charles Nutter)
@ 2013-10-01 1:23 ` headius (Charles Nutter)
2013-12-14 4:33 ` [ruby-core:59102] " nobu (Nobuyoshi Nakada)
` (2 subsequent siblings)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-10-01 1:23 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Put my patch plus a test in a PR: https://github.com/ruby/ruby/pull/408
Unfortunately the test doesn't pass, and I think it should. I'm confused as to why it fails.
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-42129
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: current: 2.1.0
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:59102] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (10 preceding siblings ...)
2013-10-01 1:23 ` [ruby-core:57514] " headius (Charles Nutter)
@ 2013-12-14 4:33 ` nobu (Nobuyoshi Nakada)
2013-12-29 16:07 ` [ruby-core:59379] " headius (Charles Nutter)
2013-12-29 16:07 ` [ruby-core:59380] " headius (Charles Nutter)
13 siblings, 0 replies; 15+ messages in thread
From: nobu (Nobuyoshi Nakada) @ 2013-12-14 4:33 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by nobu (Nobuyoshi Nakada).
My PR: https://github.com/ruby/ruby/pull/480
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-43663
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: current: 2.1.0
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:59379] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (11 preceding siblings ...)
2013-12-14 4:33 ` [ruby-core:59102] " nobu (Nobuyoshi Nakada)
@ 2013-12-29 16:07 ` headius (Charles Nutter)
2013-12-29 16:07 ` [ruby-core:59380] " headius (Charles Nutter)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-12-29 16:07 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Sorry this didn't get into 2.1 and I was unable to review. December was probably too late to get it in anyway.
Nobu's patch looks fine. If other ruby-core folks really want to keep Weakref as-is for compatibility and introduce a new type, I guess that's the way we'll have to go.
A couple comments.
* ext/weakref should be deprecated and warn when loaded once ext/weak is in place.
* Is there interest in other possible reference types? If so, having a namespace for these new references would be less cumbersome. I will describe the reference types on JVM below.
On JVM, there is WeakReference, of course. There's also two others that are useful:
* SoftReference is a reference cleared less frequently than a weak reference. This is JVM implememtation-specific, but on OpenJDK it is a combination of heap pressure (if the heap has to be expanded, soft references are cleared) or if the soft reference is not traversed for some period of time (configurable as some number of ms/MB of heap).
* PhantomReference is similar to weak reference in life cycle, but is not traversible and only useful when combined with a queue. This is lighter-weight than weak reference, since it does not have to be cleared when the object is collected; it just needs to be enqueued. It also does not impact GC cycles as much, since it does not count as a strong or weak traversible reference.
If we might want these types of references in the future, it may be good to have a Reference namespace, a la Reference::Weak, Reference::Soft, etc.
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-43944
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: 2.1.0
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
* [ruby-core:59380] [ruby-trunk - Feature #6309] Add a reference queue for weak references
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
` (12 preceding siblings ...)
2013-12-29 16:07 ` [ruby-core:59379] " headius (Charles Nutter)
@ 2013-12-29 16:07 ` headius (Charles Nutter)
13 siblings, 0 replies; 15+ messages in thread
From: headius (Charles Nutter) @ 2013-12-29 16:07 UTC (permalink / raw
To: ruby-core
Issue #6309 has been updated by headius (Charles Nutter).
Target version changed from 2.1.0 to current: 2.2.0
----------------------------------------
Feature #6309: Add a reference queue for weak references
https://bugs.ruby-lang.org/issues/6309#change-43945
Author: headius (Charles Nutter)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category:
Target version: current: 2.2.0
Most interesting uses of WeakRef are much harder to do efficiently without a reference queue.
A reference queue, as implemented by the JVM, is basically a queue into which weak references are placed some time after the object they refer to has been collected. The queue can be polled cheaply to look for collected references.
A simple example of usage can be seen in the weakling gem, with an efficient implementation of an ID hash: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
Notice the _cleanup method is called for every operation, to keep the hash clear of dead references. Failure to have a _cleanup method would mean the hash grows without bounds.
_cleanup cannot be implemented efficiently on MRI at present because there's no reference queue implementation. On MRI, _cleanup would have to perform a linear scan of all stored values periodically to search for dead references. For a heavily used hash with many live values, this becomes a very expensive operation.
It's probably possible to implement reference queues efficiently atop the new ObjectSpace::WeakMap internals, since it already keeps track of weak references and can run code when a weak reference no longer refers to a live object.
--
http://bugs.ruby-lang.org/
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2013-12-29 16:28 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-17 8:10 [ruby-core:44403] [ruby-trunk - Feature #6309][Open] Add a reference queue for weak references headius (Charles Nutter)
2012-04-19 19:51 ` [ruby-core:44470] [ruby-trunk - Feature #6309][Feedback] " mame (Yusuke Endoh)
2012-04-28 5:47 ` [ruby-core:44719] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
2012-05-03 2:29 ` [ruby-core:44826] " mame (Yusuke Endoh)
2012-05-03 2:50 ` [ruby-core:44827] [ruby-trunk - Feature #6309][Assigned] " mame (Yusuke Endoh)
2012-05-03 20:59 ` [ruby-core:44857] [ruby-trunk - Feature #6309] " headius (Charles Nutter)
2012-11-16 16:26 ` [ruby-core:49436] " headius (Charles Nutter)
2012-11-24 1:50 ` [ruby-core:49957] " mame (Yusuke Endoh)
2013-03-15 17:28 ` [ruby-core:53451] " headius (Charles Nutter)
2013-09-27 11:19 ` [ruby-core:57438] " headius (Charles Nutter)
2013-09-30 13:21 ` [ruby-core:57491] " headius (Charles Nutter)
2013-10-01 1:23 ` [ruby-core:57514] " headius (Charles Nutter)
2013-12-14 4:33 ` [ruby-core:59102] " nobu (Nobuyoshi Nakada)
2013-12-29 16:07 ` [ruby-core:59379] " headius (Charles Nutter)
2013-12-29 16:07 ` [ruby-core:59380] " headius (Charles Nutter)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).