colesbury · GitHub

Bug report

Calling sys.intern() on a string that's not interned acquires the interpreter-wide interned_mutex. This can lead to scaling bottlenecks either due to explicit sys.intern() calls or indirectly because PyObject_SetAttr calls _PyUnicode_InternMortal.

We should avoid the lock acquisition if there is an interned copy already present in the interned_dict.

For example:

_ATTR_PREFIX = "bench"
@register_benchmark
def setattr_non_interned():
    prefix = _ATTR_PREFIX
    obj = MyObject()
    for _ in range(1000 * WORK_SCALE):
        setattr(obj, f"{prefix}_a", None)
        setattr(obj, f"{prefix}_b", None)
        setattr(obj, f"{prefix}_c", None)

This came up when looking at Django scaling bottlenecks.

Linked PRs

Read the original on github.com ↗