jaltmayerpizzorno · GitHub

Summary

TMonitor.__init__ registers self.exit via atexit, and exit calls self.join() on the daemon monitor thread. If, at interpreter shutdown, the monitor thread is blocked inside

with self.tqdm_cls.get_lock():

— for example because mp_lock (the multiprocessing component of TqdmDefaultWriteLock) was left held by a forked child that died abnormally, or contended by another monitor under heavy instrumentation — then setting was_killed does not unblock the acquire, and the join waits forever. The interpreter hangs.

Reproducer (also added as a regression test)

Two monitor threads both blocked at tqdm/std.py:117 (the lock.acquire(*a, **k) in TqdmDefaultWriteLock.acquire), each trying to take the same RLock; the main thread sits in TMonitor.exitself.join(), waiting on one of those threads. py-spy excerpt from a hung interpreter:

Thread MainThread (idle):
    _wait_for_tstate_lock (threading.py:1169)
    join (threading.py:1149)
    exit (tqdm/_monitor.py:47)
Thread-3 (idle):
    acquire (tqdm/std.py:117)
    __enter__ (tqdm/std.py:124)
    run (tqdm/_monitor.py:69)
Thread-163 (idle):
    acquire (tqdm/std.py:117)
    __enter__ (tqdm/std.py:124)
    run (tqdm/_monitor.py:69)

I encountered this consistently when running tqdm's test suite under RightTyper instrumentation, which slows test execution enough to expose the race. The new test test_monitor_atexit_does_not_deadlock_on_stuck_get_lock reproduces it deterministically without needing instrumentation: it parks the lock from a setup thread, lets the monitor block on it, then asserts the captured atexit handler returns within 2s.

Fix

Split the shutdown path:

  • A new _atexit_signal() method (registered with atexit) only sets was_killed and returns. The thread is daemon, so the interpreter reaps it on shutdown without an explicit join — the join was always redundant for the atexit path.
  • exit() (called explicitly from test teardown, where joining is desirable) is unchanged.

Test plan

  • Added regression test that deterministically reproduces the deadlock with a signalling lock; verified it fails on master (2s timeout) and passes on the fix (~0.02s).
  • Full pytest tests/ passes: 148 passed, 3 skipped (147 previously + the new test).
  • No public API changes; exit() semantics for the explicit test-teardown path are unchanged.

Read the original on github.com ↗