lunixbochs · GitHub

Bug report

Bug description:

Summary

This issue is in mi_thread_data_zalloc(), triaged on commit 57d4446 by Xint Code.

The function writes td->memid = memid; and then, if !is_zero, zeroes the entire mi_thread_data_t via _mi_memzero_aligned(td, sizeof(*td)), which clears td->memid (cpython/Objects/mimalloc/init.c:227-235).

Later frees use td->memid to decide how to return memory to the OS: _mi_os_free(tdfree, sizeof(mi_thread_data_t), tdfree->memid, ...) (cpython/Objects/mimalloc/init.c:251) and in cache collection _mi_os_free(td, sizeof(mi_thread_data_t), td->memid, ...) (cpython/Objects/mimalloc/init.c:261).

If memid was zeroed, _mi_os_free_ex treats it as MI_MEM_NONE and does nothing (cpython/Objects/mimalloc/os.c:159-183), leaking the thread metadata. The enum shows MI_MEM_NONE is zero (cpython/Include/internal/mimalloc/mimalloc/types.h:425-460).

Conditions

Pre-conditions:

  • CPython built with mimalloc support (WITH_MIMALLOC) and selected at runtime (e.g., PYTHONMALLOC=mimalloc), or built with Py_GIL_DISABLED which uses mimalloc by default (cpython/Include/internal/pycore_pymem_init.h:21-38, cpython/Objects/obmalloc.c:786-793).
  • Application creates threads that perform allocations, invoking mimalloc's per-thread initialization (cpython/Objects/mimalloc/init.c:267-286).

Data flow:

  1. Thread A starts; mi_thread_data_zalloc allocates td from the OS, sets td->memid = memid, with is_zero = memid.initially_zero (likely true) and returns without zeroing (cpython/Objects/mimalloc/init.c:215-231).
  2. Thread A ends; mi_thread_data_free inserts td into the td_cache (cpython/Objects/mimalloc/init.c:239-247).
  3. Thread B starts; mi_thread_data_zalloc fetches td from td_cache with is_zero still false and zeroes the entire struct, clearing td->memid (cpython/Objects/mimalloc/init.c:200-211, cpython/Objects/mimalloc/init.c:233-235).
  4. Thread B ends; if td_cache is full, mi_thread_data_free attempts direct free: _mi_os_free(tdfree, sizeof(mi_thread_data_t), tdfree->memid, ...) with tdfree->memid == MI_MEM_NONE, so _mi_os_free_ex no-ops (cpython/Objects/mimalloc/init.c:239-251, cpython/Objects/mimalloc/os.c:159-183), leaking td.
  5. Alternatively, if kept cached, at process shutdown mi_process_done -> mi_collect(true) -> _mi_thread_data_collect tries to free cached entries with a zeroed memid, again leaking (cpython/Objects/mimalloc/init.c:595-612, cpython/Objects/mimalloc/heap.c:178-182, cpython/Objects/mimalloc/init.c:254-265, cpython/Objects/mimalloc/os.c:159-183).

Reproduction

If you run 81.py with PYTHONMALLOC=mimalloc (or a free-threaded build), it shows rss growth due to the leak.
If you run with PYTHONMALLOC=malloc, there is no rss growth shown.

Script: 81.py

% PYTHONMALLOC=malloc python3.14 81.py
PYTHONMALLOC=malloc
batches=300 width=128 total_threads=38400
start_rss=20.8 MiB
batch=  50 rss=25.6 MiB delta=4.8 MiB
batch= 100 rss=25.6 MiB delta=4.9 MiB
batch= 150 rss=25.7 MiB delta=5.0 MiB
batch= 200 rss=25.7 MiB delta=5.0 MiB
batch= 250 rss=25.7 MiB delta=5.0 MiB
batch= 300 rss=25.7 MiB delta=5.0 MiB
done
% PYTHONMALLOC=mimalloc python3.14 81.py
PYTHONMALLOC=mimalloc
batches=300 width=128 total_threads=38400
start_rss=19.7 MiB
batch=  50 rss=41.3 MiB delta=21.6 MiB
batch= 100 rss=42.7 MiB delta=23.1 MiB
batch= 150 rss=43.9 MiB delta=24.2 MiB
batch= 200 rss=45.1 MiB delta=25.4 MiB
batch= 250 rss=46.2 MiB delta=26.6 MiB
batch= 300 rss=47.4 MiB delta=27.8 MiB
done

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

Read the original on github.com ↗