Control the Inference Server

Start, configure, inspect, and stop a local Roboflow Inference Server with the inference server command.

The inference server command provides a control layer around the HTTP server that exposes Inference.

To see the details of the command, run:

inference server --help

A help guide is also available for each sub-command:

inference server start --help

inference server start

Starts a local Inference Server. It optionally takes a port number (default is 9001) and only starts the Docker container if there is not already a container running on that port.

If you would rather run your server on a virtual machine in Google Cloud or Amazon Cloud, see Deploy to Cloud.

Before you begin, ensure that you have Docker installed on your machine. Docker provides a containerized environment, allowing the Inference Server to run in a consistent and isolated manner, regardless of the host system. If you haven't installed Docker yet, you can get it from Docker's official website.

The CLI automatically detects the device you are running on and pulls the appropriate Docker image.

inference server start --port 9001 [-e {optional_path_to_file_with_env_variables}]

The parameter --env-file (or -e) is the optional path to a .env file that is loaded into your Inference Server when the values of internal parameters need to be adjusted. Any value passed explicitly as a command parameter is considered more important and shadows the value defined in the .env file under the same target variable name.

Volume mounts

Use the --volume (or -v) flag to mount a host directory into the container. This is useful for persisting files written by Workflows (for example via the local_file_sink block) to your local machine.

inference server start --volume /host/path:/container/path

You can mount multiple volumes by repeating the flag:

inference server start --volume /host/data:/data --volume /host/models:/models:ro

The optional :ro suffix mounts the volume as read-only. If omitted, the volume is mounted read-write.

Development mode

Use the --dev flag to start the Inference Server in development mode. Development mode enables the Inference Server's built-in notebook environment for easy testing and development.

inference server start --dev

Visit localhost:9001 in your browser to see the Inference landing page, which links to resources and examples. From the landing page, select "Jump Into an Inference Enabled Notebook" to open the built-in JupyterLab environment in a new tab. It comes preloaded with example notebooks and all of the dependencies needed to run Inference.

Tunnel

Use the --tunnel flag to start the Inference Server with a tunnel that exposes inference to external requests on a TLS-enabled endpoint.

The randomly generated address is shown in the server start output:

Tunnel to local inference running on https://somethingrandom-ip-192-168-0-1.roboflow.run

inference server status

Checks the status of the local Inference Server.

inference server status

inference server stop

Stops the Inference Server.

inference server stop