Bug report
Bug description:
In the free-threaded build, BytesIO.__setstate__() installs the instance __dict__ with a plain (non-atomic) store, which races the lock-free LOAD_ATTR method-cache check that reads the same instance-dict slot with an atomic acquire load.
bytesio_setstate_lock_held() sets self->dict directly,
| dict = PyTuple_GET_ITEM(state, 2); | |
| if (dict != Py_None) { | |
| if (!PyDict_Check(dict)) { | |
| PyErr_Format(PyExc_TypeError, | |
| "third item of state should be a dict, got a %.200s", | |
| Py_TYPE(dict)->tp_name); | |
| return NULL; | |
| } | |
| if (self->dict) { | |
| /* Alternatively, we could replace the internal dictionary | |
| completely. However, it seems more practical to just update it. */ | |
| if (PyDict_Update(self->dict, dict) < 0) | |
| return NULL; | |
| } | |
| else { | |
| self->dict = Py_NewRef(dict); | |
| } | |
| } |
but calling a method on the object (e.g. bio.read(...)) goes through a LOAD_ATTR method specialization that reads the instance-dict slot lock-free with an atomic acquire load.
| // _CHECK_ATTR_METHOD_LAZY_DICT | |
| { | |
| uint16_t dictoffset = read_u16(&this_instr[4].cache); | |
| char *ptr = ((char *)PyStackRef_AsPyObjectBorrow(owner)) + MANAGED_DICT_OFFSET + dictoffset; | |
| PyObject *dict = FT_ATOMIC_LOAD_PTR_ACQUIRE(*(PyObject **)ptr); | |
| if (dict != NULL) { | |
| UPDATE_MISS_STATS(LOAD_ATTR); | |
| assert(_PyOpcode_Deopt[opcode] == (LOAD_ATTR)); |
__setstate__ holds the BytesIO critical section, but the LOAD_ATTR fast path does not take the object lock, so the plain store races the atomic load on the dict slot.
Reproducer:
import io from threading import Thread, Barrier shared = io.BytesIO(b'initial payload') states = [(b'A' * 64, 0, {}), (b'B' * 128, 32, {}), (b'C' * 256, 128, {})] N_W, N_R = 1, 8 barrier = Barrier(N_W + N_R) def setter(): barrier.wait() for i in range(20000): try: shared.__setstate__(states[i % len(states)]) except Exception: pass def reader(): barrier.wait() for i in range(20000): try: shared.read(8) except Exception: pass if __name__ == "__main__": threads = [Thread(target=setter) for _ in range(N_W)] threads += [Thread(target=reader) for _ in range(N_R)] for t in threads: t.start() for t in threads: t.join()
TSAN Report:
==================
WARNING: ThreadSanitizer: data race (pid=37676)
Write of size 8 at 0x7fffb66706c8 by thread T2:
#0 bytesio_setstate_lock_held /cpython/./Modules/_io/bytesio.c:968:24
#1 bytesio_setstate /cpython/./Modules/_io/bytesio.c:980:11
#2 method_vectorcall_O /cpython/Objects/descrobject.c:476:24
#3 _PyObject_VectorcallTstate /cpython/./Include/internal/pycore_call.h:144:11
#4 PyObject_Vectorcall /cpython/Objects/call.c:327:12
#5 _Py_VectorCallInstrumentation_StackRefSteal /cpython/Python/ceval.c:768:11
#6 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:1906:35
...
Previous atomic read of size 8 at 0x7fffb66706c8 by thread T10:
#0 _Py_atomic_load_ptr_acquire /cpython/./Include/cpython/pyatomic_gcc.h:557:18
#1 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:9149:34
#2 _PyEval_EvalFrame /cpython/./Include/internal/pycore_ceval.h:122:16
...
SUMMARY: ThreadSanitizer: data race /cpython/./Modules/_io/bytesio.c:968:24 in bytesio_setstate_lock_held
==================
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux