The Control Plane of AI

Every wave of technology begins with an infrastructure gold rush.

In railroads, it was steel tracks and locomotives. In networking, it was routers and switches. In cloud, it was servers and storage.

In AI, it’s GPUs and foundation models. The money is flowing where it’s most visible: compute clusters, giant models, inference optimizers.

But history tells us the biggest long-term companies aren’t those digging for gold — they’re the ones selling the maps and controlling the flows.

The central question: Who will own the control plane of AI?

In the 2000s, Cisco and Juniper were synonymous with networking. But Nicira, co-founded by Martin Casado, showed that abstraction and programmability mattered more than proprietary boxes. Software-defined networking (SDN) virtualized the network, creating a programmable control layer hence transforming rigid infrastructure into a programmable platform.

That lesson repeated with VMware and cloud. It wasn’t the server vendors who captured the most value — it was those who defined the orchestration fabric.

Today’s AI moment looks similar: NVIDIA and OpenAI are the raw horsepower. NVIDIA provides the computational muscle and OpenAI delivers the foundational models, but the next generation of category-defining companies will be the ones building the software-defined control plane for intelligence - the very systems that make AI programmable, composable, and controllable at scale.

Let’s take a look at what I call the “control plane” of AI. Think of it as the operating system for AI in the enterprise. As shown in the image, it sits between the Model and the App layer is essentially acts as the ‘brain’ that coordinates how the models and agents are discovered, routed, governed and monitored.

It governs four critical functions:

1. Routing — Selecting the right model at the right time (cheap small model, accurate big model, domain-specific fine-tune).

   • Example: Martian and Not Diamond are experimenting with dynamic model routing frameworks.

2. Policy — Guardrails for safety, compliance, data residency, and governance.

   • Example: Lakera provides “red team as a service” and policy layers for enterprise adoption; Credo AI is the enterprise platform for AI governance helping businesses automate oversight, risk management and compliance throughout the model lifecycle.

3. Orchestration — Managing multi-agent workflows, tool usage, and long-term memory.

   • Example: LangChain / LangGraph and Cognition AI are pushing toward orchestration platforms.

4. Observability — Token level tracing, evaluation, and cost optimization.

   • Example: Weights & Biases, Humanloop, Helicone track usage and provide observability across stacks.

Together, these components form the programmable fabric that abstracts raw AI infra into usable enterprise systems.

Infra is necessary, but not sufficient. It lays the foundation, but it rarely captures enduring value once the stack matures. Three forces push the centre of gravity upward:

1. Models commoditize: The frontier of model quality is becoming somewhat flat across all the models. Open weights proliferate the market with Meta’s Llama, Mistral, xAI’s Grok and countless others. Even OpenAI, once untouchable faces margin pressure as enterprises hedge across multiple providers. The differentiation that once lived in raw model quality is eroding.

2. Serving optimizes away: Serving frameworks like vLLM, TensorRT-LLM, speculative decoding, and FP4 quantization relentlessly drive down cost and latency. What looks like a business moat today becomes a library or runtime optimisation tomorrow. Serving infrastructure is vital but its economics trend towards utility.

3. Moats migrate upward: Just as SDN shifted moats from proprietary boxes to programmable APIs, AI moats are moving from model quality to distribution, data depth, and workflow orchestration. The defensible layer is not raw horsepower, it’s the abstraction that directs and compounds it.

Infra may mint some giants - NVIDIA (already the highest market cap with over $4T) and perhaps a handful of cloud-scale providers.

But infra without control planes is just plumbing.

The real value accrues to those who define the maps, orchestrate the flows, and own the interfaces where the intelligence meets the world.

Seed/Series A: Point Solutions (building the pieces)

• Evals: Tools that test, benchmark, and score models/agents (e.x. evaluating accuracy, reliability, safety). Early-stage companies are building “the testing frameworks” for AI.

• Routing: Startups figuring out how to send each task to the right model or agent (cost, latency, accuracy tradeoffs). Think OpenRouter, Not Diamond.

• Policy Engines: Guardrail and compliance layers ensuring outputs follow rules, regulations, or enterprise policies (Credo AI, Lakera, Bedrock Guardrails).

At this stage, investors, back picks-and-shovels point solutions that address one pain point (evaluation, grounding, or safety).

Series B-D: Bundled Control Planes (stitching the pieces together)

Companies, evolve from point solutions into control planes that combine:

• Observability: See what agents models are doing, costs, compliance, debugging.

• Orchestration: Manage multi agent workflows, chaining tasks, automation.

• Routing: Decide which model or workflow runs where.

This is a phase, where startups try to own the abstraction layer above raw infra - the “operating system” for enterprise AI.

Analogous to how VMWare bundled hypervisor + management tools into a coherent control plane for compute.

Growth Stage: Category Definers (the “VMWare of AI”)

By this point, the winners have enough distribution + product breadth to become default enterprise platforms. They abrstarct away the complexity of models, infra, and compliance into one pane of glass. Just as VMWare define the control plane for virtualised servers, these companies will define the control plane of AI intelligence workflows.

So, the right analogy will not be “the next NVIDIA.” It’s “the next VMware.”

Whoever builds the AI control plane will intermediate every workflow.

Control planes win because they own the flows. They don’t just enable infrastructure; they govern how it’s used, measured, secured, and paid for.

The views expressed are those of the author and do not necessarily reflect the views of any investment firm or portfolio company.