Top Description Inners Fields Constructors Methods
org.python.modules

public Class gc

extends Object
Class Inheritance
Imports
java.util.Set, .List, .ArrayList, .IdentityHashMap, .HashSet, java.util.concurrent.atomic.AtomicBoolean, java.lang.ref.Reference, .ReferenceQueue, .WeakReference, java.lang.reflect.Array, .Field, .Modifier, org.python.core.JyAttribute, .Py, .PyException, .PyList, .PyObject, .PyInstance, .PyString, .Traverseproc, .TraverseprocDerived, .Visitproc, .Untraversable, org.python.core.finalization.FinalizeTrigger, org.python.modules._weakref.GlobalRef, .ReferenceBackend

In Jython, the gc module notably differs from that in CPython. This comes from the different ways Jython and CPython perform garbage collection. While CPython's garbage collection is based on reference counting, Jython is backed by Java's gc, which is based on a mark-and-sweep approach.

This difference becomes most notable if finalizers are involved that perform resurrection. While the resurrected object itself behaves rather similar between Jython and CPython, things are more delicate with objects that are reachable (i.e. strongly referenced) via the resurrected object exclusively. While in CPython such objects do not get their finalizers called, Jython/Java would call all their finalizers. That is because Java detects the whole unreachable subgraph as garbage and thus calls all their finalizers without any chance of direct intervention. CPython instead detects the unreachable object and calls its finalizer, which makes the object reachable again. Then all other objects are reachable from it and CPython does not treat them as garbage and does not call their finalizers at all. This further means that in Jython weak references to such indirectly resurrected objects break, while these persist in CPython.

As of Jython 2.7, the gc module offers some options to emulate CPython behavior. Especially see the flags PRESERVE_WEAKREFS_ON_RESURRECTION, DONT_FINALIZE_RESURRECTED_OBJECTS and DONT_FINALIZE_CYCLIC_GARBAGE for this.

Another difference is that CPython's gc module offers some debug features like counting of collected cyclic trash, which are hard to support by Jython. As of Jython 2.7 the introduction of a traverseproc mechanism (c.f. org.python.core.Traverseproc) made support of these features feasible. As support of these features comes with a significant emulation cost, one must explicitly tell gc to perform this. To make objects subject to cyclic trash counting, these objects must be gc-monitored in Jython. See monitorObject(PyObject), unmonitorObject(PyObject), MONITOR_GLOBAL and stopMonitoring() for this.

If at least one object is gc-monitored, collect() works synchronously in the sense that it blocks until all gc-monitored objects that are garbage actually have been collected and had their finalizers called and completed. collect() will report the number of collected objects in the same manner as in CPython, i.e. counts only those that participate in reference cycles. This allows a unified test implementation across Jython and CPython (which applies to most tests in test_gc.py). If not any object is gc-monitored, collect() just delegates to java.lang.System#gc(), runs asynchronously (i.e. non-blocking) and returns UNKNOWN_COUNT. See also DEBUG_SAVEALL for a useful gc debugging feature that is supported by Jython from version 2.7 onwards.

Implementing all these features in Jython involved a lot of synchronization logic. While care was taken to implement this without using timeouts as far as possible and rely on locks, states and system/hardware independent synchronization techniques, this was not entirely feasible.
The aspects that were only feasible using a timeout are waiting for gc to enqueue all collected objects (i.e. weak references to monitored objects that were gc'ed) to the reference queue and waiting for gc to run all PyObject finalizers.

Waiting for trash could in theory be strictly synchronized by using MXBeans, i.e. GarbageCollectionNotificationInfo and related API. However, experiments showed that the arising gc notifications do not reliably indicate when enqueuing was done for a specific gc run. We kept the experimental implementation in source code comments to allow easy reproducibility of this issue. (Note that out commented code contradicts Jython styleguide, but this one - however - is needed to document this infeasible approach and is explicitly declared accordingly).

But how is sync done now? We insert a sentinel before running gc and wait until this sentinel was collected. Timestamps are taken to give us an idea at which time scales the gc of the current JVM performs. We then wait until twice the measured time (i.e. duration from call to java.lang.System#gc() until the sentinel reference was enqueued) has passed after the last reference was enqueued by gc. While this approach is not entirely safe in theory, it passes all tests on various systems and machines we had available for testing so far. We consider it more robust than a fixed-length timeout and regard it the best known feasible compromise to emulate synchronous gc runs in Java.

The other timing-based synchronization issue - waiting for finalizers to run - is solved as follows. Since PyObject finalizers are based on org.python.core.finalization.FinalizeTriggers, Jython has full control about these finalization process from a central point. Before such a finalizer runs, it calls notifyPreFinalization() and when it is done, it calls notifyPostFinalization(). While processing of a finalizer can be of arbitrary duration, it widely holds that Java's gc thread calls the next finalizer almost instantaneously after the former. That means that a timestamp taken in notifyPreFinalization() is usually delayed only few milliseconds - often even reported as 0 milliseconds - after the last taken timestamp in notifyPostFinalization() (i.e. that was called by the previous finalizer). Jython's gc module assumes the end of Java's finalization process if postFinalizationTimeOut milliseconds passed after a call of notifyPostFinalization() without another call to notifyPreFinalization() in that time. The default value of postFinalizationTimeOut is 100, which is far larger than the usual almost-zero duration between finalizer calls.
This process can be disturbed by third-party finalizers of non-PyObjects brought into the process by external libraries. If these finalizers are of short duration (which applies to typical finalizers), one can deal with this by adjusting postFinalizationTimeOut, which was declared public for exactly this purpose. However if the external framework causing the issue is Jython aware, a cleaner solution would be to let its finalizers call notifyPreFinalization() and notifyPostFinalization() appropriately. In that case these finalizers must not terminate by throwing an exception before notifyPostFinalization() was called. This is a strict requirement, since a deadlock can be caused otherwise.

Note that the management API (c.f. com.sun.management.GarbageCollectionNotificationInfo) does not emit any notifications that allow to detect the end of the finalization phase. So this API provides no alternative to the described technique.

Usually Java's gc provides hardly any guarantee about its collection and finalization process. It not even guarantees that finalizers are called at all (c.f. http://howtodoinjava.com/2012/10/31/why-not-to-use-finalize-method-in-java). While at least the most common JVM implementations usually do call finalizers reliably under normal conditions, there still is no specific finalization order guaranteed (one might reasonably expect that this would be related to reference connection graph topology, but this appears not to be the case). However Jython now offers some functionality to compensate this situation. Via registerPreFinalizationProcess(Runnable) and registerPostFinalizationProcess(Runnable) and related methods one can now listen to beginning and end of the finalization process. Note that this functionality relies on the technique described in the former paragraph (i.e. based on calls to notifyPreFinalization() and notifyPostFinalization()) and thus underlies its unsafety, if third-party finalizers are involved. Such finalizers can cause false-positive runs of registered (pre/post) finalization processes, so this feature should be used with some care. It is recommended to use it only in such a way that false-positive runs would not cause serious harm, but only some loss in performance or so.

Nested and Inner Type Summary

Modifier and TypeClass and Description
public static class
private static class
private static class
private static class
private static class
private static class
gc.ReachableFinder

Helper to find the reachable set of an object.

private static class
private static class
private static class
private static class
gc.RefersToSetFinder

Like RefInListFinder this visitproc looks whether the traversed object refers to one of the objects in a given set.

private static class
gc.RefInListFinder

This visitproc looks whether an object refers to one of the objects in a given set.

private static class
private static class

Field Summary

Modifier and TypeField and Description
public static final String
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final PyString
public static final String
private static int
public static final int
DEBUG_COLLECTABLE

print collectable objects (in Jython scoped on monitored objects)

public static final int
DEBUG_INSTANCES

print instances (in Jython scoped on monitored objects)

public static final int
public static final int
DEBUG_OBJECTS

print other objects (in Jython scoped on monitored objects)

public static final int
DEBUG_SAVEALL

save all garbage in gc.garbage (in Jython scoped on monitored objects)

public static final int
DEBUG_STATS

print collection statistics (in Jython scoped on monitored objects)

public static final int
DEBUG_UNCOLLECTABLE

print uncollectable objects (in Jython scoped on monitored objects)

private static int
private static int
private static IdentityHashMap<PyObject, PyObject>
private static byte
private static final byte
public static final short
DONT_FINALIZE_CYCLIC_GARBAGE

CPython prior to 3.4 does not finalize cyclic garbage PyObjects, while Jython does this by default.

public static final short
DONT_FINALIZE_RESURRECTED_OBJECTS

If in CPython an object is resurrected via its finalizer and contained strong references to other objects, these are also resurrected and not finalized in CPython (as their reference count never drops to zero).

public static final short
DONT_TRAVERSE_BY_REFLECTION

Reflection-based traversal is an inefficient fallback method to traverse PyObject subtypes that don't implement org.python.core.Traverseproc and are not marked as org.python.core.Untraversable.

private static int
public static final short
public static final short
public static PyList
garbage

list of uncollectable objects

private static short
private static int
public static long
private static final AtomicBoolean
private static ReferenceQueue<Object>
private static int
public static final short
INSTANCE_TRAVERSE_BY_REFLECTION_WARNING

Makes gc emit reflection-based traversal warning for every traversed object instead of only once per class.

private static long
private static boolean
private static final byte
private static long
public static final short
MONITOR_GLOBAL

This flag tells every newly created PyObject to register for gc monitoring.

private static final Set<gc.WeakReferenceGC>
private static boolean
private static final byte
private static boolean
private static int
private static boolean
private static final List<Runnable>
private static Thread
private static final List<Runnable>
public static long
private static long
private static final List<Runnable>
private static final List<Runnable>
public static final short
PRESERVE_WEAKREFS_ON_RESURRECTION

If a PyObject is resurrected during its finalization process and was weakly referenced, Jython breaks the weak references to the resurrected PyObject by default.

private static HashSet<Class<? extends PyObject>>
private static IdentityHashMap<PyObject, PyObject>
public static final short
SUPPRESS_TRAVERSE_BY_REFLECTION_WARNING

If this flag is not set, gc warns whenever an object would be subject to reflection-based traversal.

public static final int
UNKNOWN_COUNT

A constant that can occur as result of collect() and indicates an unknown number of collected cyclic trash.

public static final short
USE_PY_WRITE_DEBUG

In Jython one usually uses Py.writeDebug for debugging output.

public static final short
public static final short
VERBOSE_COLLECT

Enables collection-related verbose output.

public static final short
VERBOSE_DELAYED

Enables delayed finalization related verbose output.

public static final short
VERBOSE_FINALIZE

Enables finalization-related verbose output.

public static final short
VERBOSE_WEAKREF

Enables weakref-related verbose output.

private static boolean

Constructor Summary

AccessConstructor and Description
public
gc()

Method Summary

Modifier and TypeMethod and Description
public static void
public static void
addJythonGCFlags(short flags)

This is a convenience method to add flags via bitwise or.

public static boolean
canLinkToPyObject(Class<?> cls, boolean actual)

This method checks via type-checking-only, whether an object of the given class can in principle hold a ref to a PyObject.

private static boolean
canLinkToPyObjectIntern(Class<?> cls, IdentityHashMap<Class<?>, Class<?>> alreadyChecked)

public static int

Returns:

Collected monitored cyclic trash objects or gc.UNKNOWN_COUNT if nothing is monitored or -1 if an error occurred and collection did not complete.
collect
(int
(ignored)
generation
)

The generation parameter is only for compatibility with CPython collect() and is ignored.

public static int

Returns:

Number of collected monitored cyclic trash objects or UNKNOWN_COUNT if nothing is monitored or -1 if an error occurred and collection did not complete.
collect
()

If no objects are monitored, this just delegates to System.gc() and returns UNKNOWN_COUNT as a non-erroneous default value.

private static int
private static List<gc.WeakReferenceGC>
public static boolean
public static boolean
public static void
disable()

Not supported by Jython.

public static void
enable()

Does nothing in Jython as Java gc is always enabled.

public static Set<PyObject>
findCyclicObjects(PyObject start)

Return objects that are reachable from start AND can reach start, thus participate in a cycle with start.

private static IdentityHashMap<PyObject, PyObject>
private static Set<PyObject>
findReachables(Iterable<PyObject> pool)

Computes the set of objects reachable from pool, not necessarily including pool itself; only those objects from pool that are reachable from at least one other object in pool will be included in the result.

public static PyObject
get_count()

Not supported by Jython.

public static int
get_debug()

Copied from CPython doc: Get the garbage collection debugging flags.

public static PyObject
get_objects()

Only works reliably if monitorGlobal is active, as it depends on monitored objects to search for referrers.

public static PyObject
get_referents(PyObject[] args, String[] kwargs)

Only works reliably if all objects in args properly implement the Traverseproc mechanism (unless reflection-based traversal is activated and works stable).

public static PyObject
get_referrers(PyObject[] args, String[] kwargs)

Only works reliably if monitorGlobal is active, as it depends on monitored objects to search for referrers.

public static PyObject
get_threshold()

Not supported by Jython.

public static short
getJythonGCFlags()

Gets the current Jython specific gc flags.

public static boolean
public static gc.WeakReferenceGC
getMonitorReference(PyObject ob)

Avoid to use this method.

public static int
public static int
public static PyObject
is_tracked(PyObject[] args, String[] kwargs)

is_tracked is - in Jython case - interpreted in the sense that gc.collect will be able to count the object as collected if it participates in a cycle.

public static boolean
isenabled()

Always returns true in Jython.

public static boolean
public static boolean
private static boolean
public static boolean
public static void
markCyclicObjects(PyObject start, boolean uncollectable)

Mark all objects that are reachable from start AND can reach start, thus participate in a cycle with start.

public static void
public static void
monitorObject(PyObject ob, boolean initString)

private static boolean
private static boolean
private static void
notifyAbortFinalize(PyObject abort, boolean cyclic)

For now this just calls notifyFinalize, as the only current purpose is to decrement the open finalizer count.

public static void
notifyFinalize(PyObject finalized)

Do not call this method manually.

public static void
public static void
protected static void
private static boolean
private static boolean
public static void
public static void
registerPostFinalizationProcess(Runnable process)

Registers a process that will be called after all finalization during gc run is done ("finalization" refers to Jython style finalizers ran by org.python.core.finalization.FinalizeTriggers; to care for other finalizers these must call gc.notifyPreFinalization() before anything else is done and gc.notifyPostFinalization() afterwards; between these calls the finalizer must not terminate by throwing an exception).

public static void
public static void
registerPreFinalizationProcess(Runnable process)

Registers a process that will be called before any finalization during gc run takes place ("finalization" refers to Jython style finalizers ran by org.python.core.finalization.FinalizeTriggers; to care for other finalizers these must call gc.notifyPreFinalization() before anything else is done and gc.notifyPostFinalization() afterwards; between these calls the finalizer must not terminate by throwing an exception).

public static void
public static void
removeJythonGCFlags(short flags)

This is a convenience method to remove flags via bitwise and-not.

private static Set<PyObject>
removeNonCyclic(Iterable<PyObject> pool)

Returns all objects from pool that are part of reference cycles as a new set.

private static IdentityHashMap<PyObject, gc.WeakReferenceGC>
removeNonCyclicWeakRefs(Iterable<gc.WeakReferenceGC> pool)

Returns all objects from pool that are part of reference cycles as a new set.

public static void
restoreFinalizer(PyObject obj)

In addition to what org.python.core.finalization.FinalizeTrigger#ensureFinalizer(PyObject) does, this method also restores the finalizer's org.python.core.finalization.FinalizeTrigger's flags by taking the values from the former finalizer.

public static void
restoreWeakReferences(PyObject rst)

Restores weak references pointing to rst.

private static void
public static void
set_debug(int flags)

Copied from CPython doc: Set the garbage collection debugging flags.

public static void
set_threshold(PyObject[] args, String[] kwargs)

Not supported by Jython.

public static void
setJythonGCFlags(short flags)

Sets the current Jython specific gc flags.

public static void
setMonitorGlobal(boolean flag)

public static void
private static void
private static void
syncCollect(int[] stat, boolean debugStat)

public static int
traverse(PyObject ob, Visitproc visit, Object arg)

Does its best to traverse the given org.python.core.PyObject ob.

public static int
traverseByReflection(Object ob, Visitproc visit, Object arg)

This method recursively traverses fields of ob.

private static int
public static void
public static boolean
public static boolean
public static void
unregisterPostFinalizationProcessAfterNextRun(Runnable process)

Useful if a process wants to remove another one or itself during its execution.

public static boolean
public static void
unregisterPreFinalizationProcessAfterNextRun(Runnable process)

Useful if a process wants to remove another one or itself during its execution.

private static void
private static void
public static void
writeDebug(String type, String msg)

Works like org.python.core.Py#writeDebug(String, String), but prints to org.python.core.Py#writeDebug(String, String) (i.e. subject to Jython's verbose level) or directly to System.err, according to USE_PY_WRITE_DEBUG.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait