Choosing the Right Tool

When to use the Inference server, the inference package, the Inference SDK, and the Inference CLI.

Inference is the open source computer vision deployment framework behind Roboflow's hosted APIs. It ships as four pieces that work together. Most projects run the server and talk to it with the SDK.

Component	What it is	When to use it
Inference server	A Docker container that exposes models and Workflows over an HTTP API.	You want one deployment that many clients, languages, or video streams can call.
inference package	The Python library that loads and runs models directly in your own process.	Lowest latency, no HTTP hop; your app is Python and runs on the same machine as the model.
inference-sdk package	A lightweight Python HTTP client for the server.	The standard way to call a local server, the Serverless Hosted API, or a Dedicated Deployment - only api_url changes.
inference-cli package	The command line tool that starts servers, runs models and Workflows, and benchmarks a deployment.	Installing and managing the server, quick one-off predictions, benchmarking.

For the architecture of a running server (how requests, video streams, and Workflows flow through it), see Inference Architecture. To install and run the server on your own hardware, start with Self-Hosted Deployment.

Example notebooks

Runnable notebooks that solve a specific problem with Inference:

• Use CLIP to classify videos - identify the similarity of frames in a video to arbitrary text prompts.

• Stream video with WebRTC - process a webcam, RTSP stream, or video file with a model or Workflow.

• RGB anomaly detection - verify that the color in a segmentation mask is correct.

The Roboflow Notebooks repository has many more training and deployment examples.

Community and source code

• Inference on GitHub - source code, issues, and feature requests. See CONTRIBUTING.md to contribute.

• Roboflow Forum - get help from and connect with a community of computer vision developers.

• Roboflow Blog - tutorials, model releases, and computer vision news.

• Supervision - reusable utilities for annotating, filtering, and tracking predictions.