PetrochukM · GitHub

DataLoader when interacting with DistributedDataParallel and tqdm==4.26.0 causes semaphores to leak.

PyTorch issue: pytorch/pytorch#9985 (comment)

Any ideas on resolving this?

from torch import multiprocessing
# DEPENDANCY: This is required for ``DistributedDataParallel``
# https://pytorch.org/docs/stable/nn.html?highlight=distributeddataparallel#torch.nn.parallel.DistributedDataParallel
try:
    multiprocessing.set_start_method('spawn')
except RuntimeError:
    pass
# DEPENDANCY: This is required for ``from tqdm import tqdm``
# https://github.com/tqdm/tqdm/blob/96d8a3c3642474144f53f74331ef2172d1c39496/tqdm/_tqdm.py#L74
mp_lock = multiprocessing.RLock()
import torch
from torch.utils.data import DataLoader
if __name__ == '__main__':
    data_iterator = torch.utils.data.DataLoader([torch.tensor(i) for i in range(10)], num_workers=4)
    for batch in data_iterator:
        pass
/Users/michaelp/.pyenv/versions/3.6.5/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/Users/michaelp/.pyenv/versions/3.6.5/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/Users/michaelp/.pyenv/versions/3.6.5/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/Users/michaelp/.pyenv/versions/3.6.5/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))

Read the original on github.com ↗