Why Graph Traversal Beats RAG for Safety-Critical OT Diagnostics

On a 10-device OT mesh, RAG hallucinated topology paths that did not exist. Not occasionally — structurally. The failure was not a prompt engineering problem or a retrieval tuning problem. It was a domain-mismatch problem, and the fix required a different architectural decision about what the LLM should be allowed to compute.

At the time, RAG was the default recommendation for grounding any LLM in domain data — the answer you reached for before knowing the question. Rejecting it for OT topology was not as obvious a call as it now appears, and the replacement — graph traversal exposed as an agent tool — was less proven than RAG itself. Choosing the newer, less-established pattern was the deliberate bet this post documents.

The standard recommendation for grounding LLMs in domain-specific data is RAG: feed your documents into a vector store, retrieve the most semantically similar chunks at query time, append them to the prompt. This was not one option among equals — in the conversations and literature I was tracking at the time, it was the near-default starting point for any team working on LLM grounding. For document-centric domains — policy lookups, FAQ systems, product documentation — it works well.

We tested it for OT network topology queries. It produced wrong answers. Here is the mechanism.

The Test#

Digital Coworker uses a topology agent to answer questions about OT network connectivity. A representative query: “Is device A connected to device B through switch C, and if so, through which ports?” The answer determines whether a redundant ring path exists — the first step in diagnosing an MRP ring reconfiguration event.

Two retrieval approaches, same 10-device mesh network:

RAG approach: Network topology described in structured documents — switch configurations, LLDP neighbor tables, port assignments. Embedded using Amazon Bedrock’s embedding model, stored in a vector knowledge base. At query time, relevant chunks retrieved by semantic similarity and appended to the agent prompt.

Graph traversal approach: Network topology represented as a directed graph using NetworkX. Each device is a node; each physical connection is an edge with port metadata. At query time, the agent called a graph traversal tool that ran a path query directly on the live graph structure.

What Happened#

The RAG approach returned incorrect connection paths in two failure modes:

Failure mode 1 — semantic confusion between similar configurations. Two switches with the same VLAN assignments, same port counts, and similar naming conventions produced configuration documents with high semantic similarity. Retrieval occasionally surfaced the wrong switch’s configuration for a given query. The LLM constructed a path using configuration data from a device not actually in the queried path.

Failure mode 2 — incomplete path reconstruction. For queries spanning network segments not represented explicitly in any single document — the path from a PLC in segment A to an HMI in segment B through two intermediate switches — RAG retrieved the best-matching individual segments. The LLM stitched them together using inference about how they might connect. The result was a plausible but incorrect path.

Both failures are structural. Network topology is relational — the truth of a connection query depends on graph structure, not on the semantic content of any individual document. Embedding a topology document into a vector space loses the graph. Semantic similarity is not structural similarity.

The Correct Architecture for Structural Retrieval#

Graph traversal does not have these failure modes because it does not retrieve documents — it queries structure. A path query on a NetworkX graph either finds a verified path or returns no path. No probabilistic confidence score, no retrieval ranking, no opportunity for a plausible-but-incorrect inference. The graph is the ground truth within polling cycle freshness; the traversal is deterministic. Committing to this approach meant accepting that agent tool-use for deterministic traversal was a newer, less-established pattern than RAG — the trade was determinism in exchange for building on ground that had less operational precedent at the time.

The implementation uses a graph built from live device state — LLDP neighbor discovery data polled directly from the OT network — not from documents. When a cable is physically moved or a port is reconfigured, the next polling cycle updates the graph. The agent’s answers are grounded in current state, not archived documentation.

The knowledge base of 152 analysed customer projects covering more than 400 OT device types was used to validate the failure pattern taxonomy — not as a retrieval corpus. It feeds the agent’s understanding of what MRP failure signatures look like; the graph provides the live topology the agent queries when diagnosing a specific event.

When This Principle Generalises#

The architectural choice between RAG and deterministic retrieval is a function of the domain’s answer type:

Use RAG when: The correct answer is contained in a document or passage, the query is about content or meaning, and a plausible-but-imprecise answer is acceptable. Support documentation, policy interpretation, product specification lookup.

Use deterministic retrieval when: The correct answer depends on relationships between entities, the query is about structure or state, and an incorrect answer has real consequences. Network topology, database schema relationships, dependency graphs, process flow validation, regulatory compliance checks where the answer is a deterministic yes/no against a rule graph or checklist — not against interpretive policy text.

In safety-critical environments, the second category is substantially larger than in consumer or enterprise software contexts. The tolerance for a plausible-but-incorrect answer is lower when the downstream action is a configuration recommendation for production OT equipment.

The anti-hallucination architecture for Digital Coworker is not a special case. It is the right answer for deploying AI in safety-critical environments: identify the queries where the answer is structural, and use structural retrieval for those queries. Reserve RAG for queries where semantic similarity is the right measure of relevance. The discipline this requires is resisting the default — knowing when a widely recommended, general-purpose tool is the wrong tool for your specific domain.

This post is part of the Digital Coworker series — the architectural decisions behind Digital Coworker, Belden’s first AI-native industrial product.

The broader architectural context — four observable markers of AI-native design — is covered in: What “AI-Native” Actually Means in an Industrial Product.

The organisational reflection that pairs with this post: When the Algorithm Admits It Doesn’t Know the Path.