GPU Docker Monitor

Motivation

When running multiple Docker containers with GPU access, nvidia-smi shows GPU processes but doesn't indicate which Docker container each process belongs to. This makes it difficult to manage GPU memory - if you need to free up VRAM, you can't easily identify which container to stop or restart. This script solves that problem by mapping GPU processes to their corresponding Docker containers.

Overview

A simple bash script that maps GPU processes to their Docker containers, showing which containers are using GPU resources.

Features

Maps GPU process PIDs to their Docker container names
Shows GPU framebuffer memory usage (MB) per process, including display/render clients (not just CUDA compute processes)
Labels non-Docker processes as (host)
Sorts output by descending VRAM usage
Automatically detects whether sudo is needed for Docker commands
Clean tabular output for easy reading

Requirements

nvidia-smi (NVIDIA GPU drivers)
docker or sudo docker access
bash shell

Usage

Simply run the script:

./gpu_docker_monitor.sh

Example output:

PID       DOCKER_CONTAINER              GPU_MEM_MB  PROCESS_CMD
--------  ----------------------------  ----------  ------------------
2949447   kokoro                        964         python3
2892405   llama_cpp                     358         llama-server
1681872   (host)                        214         Proton
1858974   (host)                        12          Xorg

How It Works

The script:

Uses nvidia-smi pmon -s m to get all GPU processes with their framebuffer memory usage in MB (covers CUDA, display, and render clients)
Reads /proc/<pid>/cgroup to identify which Docker container each process belongs to without spawning a docker top per container
Labels non-Docker processes as (host)
Displays results sorted by descending VRAM usage in a formatted table

License

This project is licensed under the AGPLv3 license.

Development

This tool was developed with assistance from Claude Code.