Feature or enhancement
Proposal:
We can improve performance of cpython by using stealing methods (_PyList_AppendTakeRef, _PyDict_SetItem_Take2, PyTuple_FromPairSteal etc.) instead of the normal ones. Using stealing methods we typically avoid an incref/decref pair and sometimes error handling is simplified. We should only make changes where it makes a performance impact, the code is not more complex and the modules or methods affected are performance critical.
A list of possible cases (there might be more). The issue is open for anyone to work on.
| Location | Module / context | Helper |
|---|---|---|
Modules/_csv.c:688 |
csv reader, per field | _PyList_AppendTakeRef |
Modules/pyexpat.c:472 |
pyexpat StartElement, per attribute | _PyDict_SetItem_Take2 |
Modules/_elementtree.c:3351 |
ElementTree C accel attrib dict | _PyDict_SetItem_Take2 |
Modules/_io/bytesio.c:577 |
io.BytesIO.readlines | _PyList_AppendTakeRef |
Modules/_io/iobase.c:756 |
io.IOBase.readlines (generic) | _PyList_AppendTakeRef |
Objects/frameobject.c:639 & 658 |
frame f_locals / vars() / tracing |
_PyList_AppendTakeRef |
Modules/_pickle.c:5138 |
pickle Pickler memo copy | _PyDict_SetItem_Take2 |
Modules/_io/textio.c:2337 & 2343 |
io.TextIOWrapper readline (multi-chunk) | _PyList_AppendTakeRef |
Modules/_asynciomodule.c:1213 |
asyncio future done-callback compaction | _PyList_AppendTakeRef |
Linked PRs
- gh-150942: Speed up re.findall and re.sub/subn result building #150943
- gh-150942: Speed up json.loads array and object decoding #150945
- gh-150942: Speed up csv.reader row building #150995
- gh-150942: Speed up frame local item collection #151002
- gh-151059: Use PyObject_CallMethodOneArg in datetime call_tzinfo_method #151062
- gh-150942: Speed up ElementTree attribute parsing #151209
- gh-150942: Optimize PicklerMemoProxy.copy() with _PyDict_SetItem_Take2 #151608
- gh-150942: Speed up asyncio Future.remove_done_callback #153013
- gh-150942: Speed up pyexpat attribute parsing #154860
- gh-150942: Speed up BytesIO.readlines() using reference-stealing append #155171
- gh-150942: Speed up IOBase.readlines #155304
- gh-150942: Optimize stringlib split/splitlines with _PyList_AppendTakeRef #155922
- gh-150942: Speed up re.split result building #155940
- gh-150942: Improve performance of csv reader by using _PyList_AppendT… #155957