Self-Hosted Deployment of ScaleDown

The managed ScaleDown API handles most use cases. But some organizations have requirements that no external API can satisfy regardless of its security posture: data that cannot leave the network boundary, air-gapped environments, regulatory regimes that mandate on-premise processing, or latency constraints that require co-location with the application layer.

Self-hosted ScaleDown exists for these cases. The models run on your GPUs, the API runs on your servers, and no data leaves your environment. There is no callback to ScaleDown’s infrastructure, no telemetry sent externally, no license server that requires an outbound connection.

A self-hosted deployment includes the full ScaleDown model suite packaged as container images. The deployment is a standard containerized service: pull the image, configure the runtime, expose the API. The interface is identical to the managed API, so existing client code works without modification.

The models are optimized for inference on commodity GPU hardware. Minimum requirements depend on which models you deploy and your throughput targets, but a L40 GPU node can serve all four model types for moderate workloads. Horizontal scaling follows the same patterns as any stateless inference service: add nodes behind a load balancer.

ScaleDown provides the container images and deployment manifests. Your team handles the infrastructure: compute provisioning, networking, monitoring, and scaling. We provide reference architectures for common setups (Kubernetes with GPU node pools, standalone Docker on bare metal, ECS/EKS on AWS) and engineering support during initial deployment.

Updates are delivered as new container image versions. You control the rollout schedule, and if a specific model version is validated against your compliance requirements, you can pin it indefinitely.

The API contract is identical between managed and self-hosted. Endpoints, request formats, response formats, and model behavior are the same. If you build against the managed API and later migrate to self-hosted (or vice versa), the only change is the base URL.

Model quality is identical because the models are identical. The self-hosted images contain the same weights, the same tokenizers, and the same inference logic as the managed service.

You take on operational responsibility: uptime, scaling, GPU utilization, model updates. ScaleDown provides the models and support. For organizations with existing ML infrastructure teams, this is a natural fit. For organizations without GPU operations experience, the managed API is typically the better starting point.

Licensing is per-deployment rather than per-token. Contact us for pricing based on your scale and deployment configuration.

Self-hosted ScaleDown gives you the same models and API with full data sovereignty. Nothing leaves your network. No external dependencies at runtime. You control the hardware, the update schedule, and the network boundary.

We offer 50M free tokens for every agent on the managed API. For self-hosted deployments, reach out at scaledown.ai.