Keep a workload correct and available when its preferred compute is unobtainable—by descending an ordered ladder of substitution rungs until one is obtainable, and climbing back up when the preferred rung frees.

The descent is only half the pattern. The climb-back (promotion) is what makes this a supply pattern rather than a one-way degradation slide: when the preferred rung's capacity returns, the workload migrates back up under policy. A "ladder" reads as purely downward—resist that framing; the promotion arm is load-bearing.

This is a small, vendor-neutral, dependency-free reference implementation—a teaching artifact you can read in one sitting and pip install, not a framework. It is not a scheduler, a control plane, or a Kubernetes operator. Cloud products appear here only as examples of the pattern, never as dependencies of the core.

This is a reference implementation. It ships no benchmarks, claims no production adoption, and invents no performance numbers. Its job is to make the pattern concrete in code.

Why this exists: Compute Scarcity Is Permanent. Build a Ladder. makes the case for the pattern. This repo is the runnable companion. Read the post for the argument; read the code here to use it.

The pattern in 20 lines

from compute_fallback_ladder import (
    Ladder, Policy, Requirement, Rung, Selector, InMemoryProbe,
)
ladder = Ladder(rungs=[
    Rung("preferred", Requirement("H100", 8, 640), cost=100.0, quality=1.0),
    Rung("alternate", Requirement("A100", 8, 320), cost=60.0,  quality=0.9),
    Rung("cpu",       Requirement(None,  0, 64),   cost=5.0,   quality=0.4),  # floor: always serves
], policy=Policy(promotion_hysteresis=2))
probe = InMemoryProbe(available={"H100": 8, "A100": 8})
selector = Selector(ladder, probe, on_transition=print)
selector.select()                 # -> preferred (highest obtainable now)
probe.set_available("H100", 0)    # preferred capacity withdrawn
selector.select()                 # -> alternate (immediate descent)
probe.set_available("H100", 8)    # capacity returns...
selector.select(); selector.select()  # -> preferred (promoted after hysteresis clears)

Run the 30-second demo without writing any code:

$ python -m examples.local.demo

The core abstraction

Install

$ pip install compute-fallback-ladder      # core: zero runtime dependencies
$ pip install 'compute-fallback-ladder[ml]' # optional: the real-model ml example

Requires Python 3.12+.

The pattern, shipping: the GKE ComputeClass CRD

The whole pattern already exists as a product primitive. A GKE custom ComputeClass is a ladder you declare instead of build—the ordered rungs, the floor behavior, and the climb-back are all fields:

apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: fallback-ladder-example
spec:
  priorities: # THE LADDER: ordered rungs, tried top to bottom
    - machineType: a3-highgpu-8g # rung 0: 8x H100, the preferred rung
    - machineType: a2-highgpu-8g # rung 1: 8x A100
    - machineType: g2-standard-48 # rung 2: L4
    - machineFamily: n4 # floor: CPU-only, broadly available
  whenUnsatisfiable: ScaleUpAnyway # THE FLOOR: descend rather than place nothing
  nodePoolAutoCreation:
    enabled: true # let the autoscaler build a pool for the rung it lands on
  activeMigration:
    optimizeRulePriority: true # PROMOTION: climb back when a better rung frees

Three fields carry the pattern, and two of them are traps if you leave them out:

Omitting whenUnsatisfiable inverts the pattern: the floor becomes "serve nothing." Omitting activeMigration deletes half of it: you get a slide, not a ladder.

This was verified on a live GKE cluster, not asserted. Both failure modes above were observed before they were documented, and a full climb-back—provision the higher rung, cordon, drain, migrate the Pod up, delete the drained node—took 5m41s end to end. The session logs and the exact commands are in examples/gke-computeclasses/VERIFIED.md.

The honest caveat, carried verbatim from the GKE docs: "Custom ComputeClasses influence autoscaling decisions but are not considered by kube-scheduler." This is the autoscaling arm of the ladder, not a per-Pod scheduling guarantee.

Google Cloud appears here as an example of the pattern shipping in a real product, never as a dependency of the core library—which stays vendor-neutral and dependency-free.

Examples

Each example actually runs, and the local + kubernetes examples are exercised end to end in CI.

• examples/local/—pure-fake probe, zero cloud. Watch the ladder walk down as capacity is withdrawn and climb back when it returns.
• examples/kubernetes/—the real KubernetesProbe against a kind/minikube cluster. Vendor-neutral, reproducible on a laptop.
• examples/gke-computeclasses/—the worked, fully annotated manifest behind the section above, plus the live verification record.
• examples/ml_inference/—the motivating case: preferred accelerator → alternate → quantized/smaller model → CPU → cached answer, with the quality-loss trap demonstrated on a genuinely small open model.

Cloud Mechanism Reference

The pattern is cloud-agnostic, and the post argues why in The Mechanisms Already Ship. This table is the reference you come back to: each cloud already provides mechanisms you wire into a CapacityProbe and a rung ladder, linked to its primary documentation.

Lineage

This pattern extends known work, and the post traces the ancestry in What This Stands On: opportunistic computing (Condor, ICDCS 1988), backfill scheduling (Mu'alem & Feitelson, IEEE TPDS 2001), and brownout / graceful degradation (Klein et al., ICSE 2014), with the Circuit Breaker (Nygard, Release It!, 2007) as its sibling—the breaker sheds demand where the ladder re-sources supply.

Development

$ pip install -e '.[dev]'
$ pytest

The Kubernetes example additionally needs Docker, kind, and kubectl; see examples/kubernetes/README.md.

License

Apache-2.0. See LICENSE.