colesbury · GitHub

Bug report

Background

The free threading GC uses two queue-like data structures to keep track of objects:

  • struct worklist, which is a singly linked list that repurposes ob_tid for the linked list pointer

  • _PyObjectStack, which is effectively a dynamically sized array of PyObject*. (Implemented using a linked list of fixed size array buffers).

The struct worklist data structure is convenient because enqueueing objects doesn't require a memory allocation and so can't fail. However, an object can only be part of one "worklist" at a time, because each object has only one ob_tid field.

Bug

Other threads can run while the GC is finalizing cyclic garbage and while it's calling tp_clear() and other clean-up.

During that time, some thread may call gc.get_objects(), which can return otherwise "unreachable" objects. The implementation of _PyGC_GetObjects temporarily pushes objects to a struct worklist, including objects that might already be part of some other worklist, overwriting the linked list pointer. This essentially corrupts the state of the in-progress GC and causes assertion failures.

Proposed fix

  • We should probably exclude objects in the "unreachable" (i.e. _PyGC_BITS_UNREACHABLE) from being returned by gc.get_objects()
  • We should limit the use of struct worklist to the actual GC and use _PyObjectStack (or some other data structure) in _PyGC_GetObjects(). This reduces the risk of bugs causing an object to be added to more than one worklist.

Linked PRs

Read the original on github.com ↗