The Data Layer Is the Epistemic Ground for Agents

The three specialised agents in Digital Coworker — VLAN, MRP, and Network Analysis — exchange no private snapshot of network state: no agent’s cached view of the network becomes another agent’s premise. Coordination over network state runs through the data layer. Where a query needs another agent’s domain judgment rather than shared state, the Network Analysis Agent invokes VLAN or MRP directly as a Bedrock collaborator — the mechanism Part 3 covers. This post is about the state those invocations, and every other query, ultimately agree on.

That design choice has consequences that surface only when agents reason across domain boundaries.

The Consistency Problem Is Not an Agent Problem#

If the VLAN Agent and the Network Analysis Agent both reason about Switch-Core-3, they had better agree on what it looks like: same VLAN memberships, same interface states, same topology position.

If each agent maintains its own private snapshot — cached at query time, potentially stale by different amounts — they reach conclusions from divergent premises. The Network Analysis Agent correlating an MRP ring anomaly with a VLAN boundary condition operates on data that may be hours older than what the VLAN Agent just queried. A human operator following both recommendations simultaneously faces conflicting instructions.

The instinct is to add coordination: a message bus, a shared cache, a consistency protocol. The decision in Digital Coworker was the opposite. Add nothing. Ground all agent reasoning in a single shared data layer.

When every agent queries the same Apache Iceberg tables, consistency is a structural property, not a runtime guarantee maintained by agent cooperation. The agents cannot diverge on network state because they are not maintaining network state. The lakehouse is.

The question “why Iceberg when you already have databases” came up during architecture evaluation at Belden — teams using MySQL were skeptical. The answer runs through four criteria MySQL and ClickHouse could not satisfy simultaneously:

Longevity. OT solutions have 8–10 year lifetimes. Apache Iceberg is an open specification; the compute engine is replaceable without migrating the data.

Time travel for auditability. When the VLAN Agent recommended isolating a port at 09:17 and the operator applied that recommendation at 09:43, the network state at 09:17 must still be directly queryable — not reconstructed from logs. Iceberg’s immutable snapshot model provides this structurally.

Economics at per-customer scale. Digital Coworker deploys single-tenant, per-customer. One MySQL instance per customer is not viable; a shared instance violates data isolation. Iceberg on S3 in a per-customer AWS account scales correctly.

On-premises flexibility. Energy sector customers require air-gapped deployments. Iceberg plus object storage (AWS S3 for cloud, local object storage on-premises) is the best-suited option here that works with the same data format in both environments.

The formal ADR recorded the decision. The consequence accepted: storage, format, and compute are three separate components. That complexity is a drag for rapid PoC iteration — for a production OT system with a decade-long lifecycle, it is the correct trade.

What the Ingestion Pipeline Actually Does#

The agents do not query raw device telemetry. They query normalised, validated data that has already passed through an ingestion pipeline built by Arnau Martinez at Abylsen — Mario — which operates before anything reaches the Iceberg tables.

Mario’s responsibilities:

• Validate at ingest. Missing MRP state fields on certain firmware versions, absent VLAN data on specific HiOS platform configurations — caught and flagged rather than silently stored.
• Normalise across vendors. 400 devices, multiple vendors, SNMP and vendor REST API feeds, all normalised to a consistent schema before storage.
• Human override channel. Network administrators can inject corrected values when device-reported data is known to be incorrect — expert judgment as a structural input, not a bypass.

The downstream consequence is that the agents operate on ground truth. When the Network Analysis Agent runs a cross-device correlation, every device in that query has been through the same pipeline.

The Cross-Domain Query and Why It Works#

The most demanding case is not the single-domain fault. It is the fault that spans domains: an MRP ring reconfiguration that shifts VLAN broadcast traffic, triggering a threshold alert the predictive maintenance model misinterprets as optical degradation.

The Network Analysis Agent handles cross-domain correlations. For the network state it correlates — MRP ring state, VLAN boundary conditions — it does not depend on another agent’s private view of that state: it reads the same committed Iceberg snapshot the domain-specific agents read. Where the query needs a domain agent’s specialised diagnostic judgment rather than shared state, the Network Analysis Agent delegates to that agent as a Bedrock collaborator — the orchestration path detailed in Part 3. Either way, the state underneath is the one lakehouse, not a value handed agent-to-agent.

Agent analytical conclusions — diagnostic results, correlation findings — are written back to Iceberg as new records. This write path is scoped to the agents’ own output layer; the read-only posture established in Part 4 applies to OT network state, not to agent-generated analytical records. The Network Analysis Agent’s cross-domain query operates on data that is consistent by construction: same normalisation pipeline, same schema, same snapshot model. There is no consensus mechanism between agents. There is one lakehouse, and each agent reads from a committed, consistent snapshot of it — not a private divergent cache.

Auditability as a First-Order Requirement#

In OT, the recommendation has real consequences. Replacing an optical receiver costs $15,000. The human operator approving the action needs to examine the evidence.

Iceberg’s time travel provides this structurally. When the Network Analysis Agent ran at 14:32 on Tuesday, the snapshot it queried still exists — provided the snapshot retention window covers that period. Iceberg’s history.expire.max-snapshot-age-ms and history.expire.min-snapshots-to-keep table properties together control how far back snapshots are preserved, and expiry runs only when explicitly triggered. For audit-critical workloads, configure both properties to match the review SLA before any expiry job is scheduled. An engineer reviewing on Thursday runs the same query against the same snapshot — not a log replay, the exact data state the agent saw. The recommendation is reproducible.

For the full treatment of the snapshot model, schema evolution, and the emerging v4 proposals, the Apache Iceberg for Industrial OT series covers the mechanics at the metadata level.

The Architecture Claim#

The agents in Digital Coworker are not trustworthy because they are well-prompted. They are trustworthy because the data they reason over was validated before reaching them, they share one lakehouse instead of three private caches, and every recommendation is reproducible against the exact snapshot the agent queried — within the configured snapshot retention window.

The Iceberg-over-MySQL decision was a data-architecture decision. It turned out to be an agent-trustworthiness decision. The two are the same question.

The team reflection on what it means to maintain shared truth across a multi-agent team — and what it requires of the humans who design one — is on javatask.systems: Shared Truth Is Infrastructure.

The data layer in this architecture is covered in depth in Iceberg for OT, Part 5: The AI Agent Data Layer — time travel, snapshot model, v3 row lineage, and what those early v4 proposals could mean for multimodal industrial AI.

The three-agent decomposition this data layer supports is in Three Specialised Agents Beat One Generalist for OT Diagnostics.

This post is part of the Digital Coworker series.