provinzkraut · GitHub

When using msgspec.Meta(pattern="something"), msgspec creates a leaky reference to an re.Pattern, by increasing its reference count too many times.

This is caused by the fact that it creates the Pattern object with a call to PyObject_CallOneArg, which returns a new strong reference. Msgspec then add this to the Meta object, during which the Pattern's refcount is increased again. Due to this, its refcount can never fall to 0, keeping it alive indefinitely.

We can show this with a simple script:

import gc
import re
import secrets
import msgspec
string_pattern = secrets.token_hex()
msgspec.Meta(pattern=string_pattern)
re.purge()
print(
    any(o for o in gc.get_objects() if isinstance(o, re.Pattern) and o.pattern == string_pattern)
)

this should print False, since there should be no such re.Pattern objects arounds anymore, but this is not the case.


I have fixed this by foregoing the SET_FIELD macro, and instead simply assigning the field directly, without calling Py_XINCREF on its value.

Read the original on github.com ↗