The Software-Defined Industrial Edge, Part 5: The 88% Wall — The Agent Harness as the Missing Production Layer

IDC research finds up to ~88% of enterprise AI initiatives fail to reach production (IDC, CIO Playbook 2025). Enterprise agentic programs follow the same pattern: funded pilots that stall before they become operational systems.

The standard explanation is model quality. The model hallucinated. The model lost the thread. The model did not understand the domain. This explanation is comfortable because it is partly true, and because it suggests a clear solution — wait for the next model release. It is also wrong as a diagnosis of the production gap.

The concrete instantiation shipped first: the e-Kanban agent’s BeforeToolCallEvent provenance gate, bounded belief update, and Episode lineage are this harness pattern in production form (FTPFI Part 4).

The gap is a harness problem. IDC’s own root-cause analysis points to data infrastructure gaps, skills shortages, and organizational readiness as the underlying drivers — each converges at the same architectural choke point: the absence of a deterministic governance layer that makes agent behavior production-grade. The “harness is why pilots die” framing is the author’s architectural synthesis of those converging conditions, not IDC’s direct conclusion.

Agency is a product of two terms, not one#

The relationship between model capability and reliable autonomous behavior is a system equation, not a capability checklist:

Agency = Model Capability × Deterministic Harness

The multiplication matters. A capable model running without a harness produces probabilistic, context-sensitive outputs that cannot be made predictable at the system boundary. A harness without a capable model produces nothing useful. The two terms are not additive — they compound. Zero on either collapses the output.

The harness is the deterministic control plane that wraps the probabilistic model. It enforces execution boundaries, governs what context the model sees at each step, and validates actions before they reach a physical system or a write-capable API. In the industrial edge context — where an agent might issue a shutdown command, modify a maintenance schedule, or adjust a PID setpoint — the harness is not optional scaffolding. It is the mechanism that makes the difference between a working demonstration and a deployable system.

The three harness primitives below carry most of the production reliability weight. They are not theoretical — they correspond to the failure modes that account for the 88% figure: unverifiable action sequences, context corruption, and unconstrained retrieval.

Primitive 1: The read-only governance gate#

Every tool call the agent proposes must pass a policy gate before execution. The gate has one job: check the tool’s risk classification and either permit, queue for human approval, or block. It is not a prompt instruction the model can reason around. It is an interception layer between the model’s output and the tool runtime.

# harness/governance-gate.yaml
gate:
  default_policy: block          # unknown tools are blocked by default
  rules:
    - pattern: "sensor.read.*"
      action: allow
    - pattern: "actuator.*"
      action: require_approval
      approver: "plant-supervisor"
      timeout_seconds: 30
      fallback_on_timeout: block
    - pattern: "*.delete"
      action: block
    - pattern: "*.write"
      action: require_approval
      approver: "plant-supervisor"
      timeout_seconds: 60
      fallback_on_timeout: block
  audit_log: "/var/log/agent/gate-audit.jsonl"

The model may reason that deleting a stale record is the correct next action. The gate blocks the call and returns a structured rejection the model can incorporate into its next reasoning step — without ever having reached the target system. For environments operating under IEC 62443 security requirements, the gate’s audit log provides the action-level trace that satisfies the accountability requirement for autonomous systems operating in the safety-relevant zone.

The gate’s classification scheme maps directly to the operational taxonomy of the facility. Read operations on sensor data are low-risk and are permitted without delay. Actuator commands and write operations require a named human approver. Destructive operations are blocked at the class level regardless of context.

Primitive 2: The Plan-Execute-Verify loop#

Unconstrained agents exhibit a failure mode the literature describes as a hallucination cascade: an unverified intermediate error propagates through a multi-step workflow until the final output is wrong in a way that is difficult to trace back. The Plan-Execute-Verify (PEV) loop interrupts this by inserting a verification gate between each phase.

def pev_loop(objective: str, harness: AgentHarness) -> Result:
    # Phase 1 — Plan: model sees only objective + current system snapshot
    plan = harness.model.plan(
        objective=objective,
        context=harness.context.load_minimal()
    )
    harness.state.write("plan.md", plan)

    # Phase 2 — Execute: each step passes through the governance gate
    for step in plan.steps:
        step_context = harness.context.load_for(step)   # step-scoped, not session-wide
        action = harness.model.execute_step(step, context=step_context)
        result = harness.gate.submit(action)             # gate intercepts here
        harness.state.append("execution-log.jsonl", result)

        if result.status in ("blocked", "timeout"):
            harness.state.flag_for_human_review(step, result)
            break   # halt the loop; human reviews before continuation

    # Phase 3 — Verify: model reads plan vs. log, not the full session
    verification = harness.model.verify(
        plan=harness.state.read("plan.md"),
        log=harness.state.read("execution-log.jsonl")
    )
    return verification

The planning phase receives a minimal context slice: the objective and a current system state snapshot. The execution phase receives only the context relevant to the current step, retrieved by the constrained context graph (Primitive 3). The verification phase receives the original plan and the execution log — nothing else. This staged context discipline prevents the verification model from being contaminated by intermediate noise accumulated during execution.

If any step is blocked or times out, the loop halts and flags the step for human review before resuming. This is the human-on-the-loop boundary made explicit in code, not in a system prompt.

Primitive 3: The constrained context graph#

The most common mistake in moving from a RAG prototype to a production agent is treating context delivery as a bulk operation: retrieve everything semantically similar, concatenate it, inject it into the prompt. This fails for two reasons that interact.

First, attention dilution: as context density increases, a model’s ability to follow instructions placed in the middle of a long sequence degrades measurably. Second, on an edge node running local 8B-parameter inference, the token budget is fixed and inelastic.

The alternative is a constrained context graph — a structured retrieval layer that delivers the minimum context slice sufficient for the current reasoning step.

def load_for(step: PlanStep) -> Context:
    """Step-scoped context retrieval — not session-wide."""
    nodes = context_graph.traverse(
        entry=step.primary_entity,
        depth=3,            # cap at 3 hops; beyond this, result-set growth
                            # outpaces accuracy gain on multi-hop queries
        filters={
            "classification": "operational",
            "freshness_hours": 24,
            "source_verified": True
        }
    )
    return Context(
        tokens=nodes.to_prompt(budget=2048),   # hard cap per step
        provenance=nodes.lineage()             # retained for gate audit correlation
    )

The depth-3 cap is not arbitrary. Graph traversal practice and in-memory database benchmarks establish that traversals beyond 3–4 hops produce exponential result-set growth with diminishing accuracy returns on queries requiring multi-entity reasoning. A context graph with a hard depth limit and a token budget is a different artifact from a vector store with a similarity threshold — it is a bounded, inspectable retrieval policy.

Agent memory itself can be persisted as an Apache Iceberg table: time-stamped entity nodes and relationship edges written to the lakehouse layer, with time-travel queries for audit and reproducible replay. Part 6 of this series addresses the data substrate — the Unified Namespace and Iceberg lakehouse that both analytics pipelines and agent memory share as the queryable layer beneath everything else.

Running the harness on the sovereign substrate#

The three primitives above are software. They require a runtime. On the industrial edge, that runtime is the same Quadlet substrate this series has built from Part 1 onward — a systemd unit managing the harness process alongside the local-LLM serving layer introduced in Part 4.

# /etc/containers/systemd/agent-harness.container
[Unit]
Description=Agent Harness
After=local-llm.service         # model must be ready before the harness starts

[Container]
Image=registry.internal/agent-harness:v1.2.3
Volume=/var/log/agent:/var/log/agent:rw
Volume=/etc/agent/governance-gate.yaml:/etc/harness/gate.yaml:ro  # gate config is read-only to the container

[Service]
Restart=on-failure
RestartSec=10s

The governance-gate configuration mounts read-only: the harness process can read its policy but cannot modify it at runtime. The model endpoint is provided by the local-LLM service declared in After=. If the model restarts — due to an update, an OOM kill, or a hardware event — the harness restarts after it. This ordering is not incidental. The PEV loop requires predictable startup sequencing; systemd provides it without a Kubernetes control plane.

The Model Context Protocol (MCP), an anchor project of the Agentic AI Foundation (AAIF) under the Linux Foundation (formed December 2025), is the interface between the harness and the model’s tool-call mechanism. The governance gate sits between the MCP dispatcher and the tool runtime — it intercepts the tool-call request at the protocol boundary before it reaches the execution environment. This positioning makes the gate portable: any MCP-compliant model endpoint — local GGUF inference, a remote API, or a hybrid topology — passes through the same interception point.

KitOps OCI ModelKits, covered in Part 3 of this series, provide the version-pinning property the harness needs to make its audit log meaningful. When the gate records a blocked action, the model checkpoint that generated the request must be reproducible. A ModelKit artifact in the OCI registry gives that guarantee without requiring a separate model registry infrastructure.

The compounding argument#

The sovereign substrate — Quadlets, Margo, local inference — is the necessary condition: it gives the agentic layer a runtime that operates without WAN dependency, survives air-gap constraints, and does not require an orchestration platform that OT teams cannot staff.

The harness is the sufficient condition: it gives the model a deterministic operational envelope that makes its probabilistic outputs safe to act on in a physical environment.

Neither produces reliable autonomous behavior on its own. The 88% production failure rate is not an argument against deploying agents on the industrial edge. It is a specification: production-grade agency requires a deterministic operational layer around the model. That layer — governance gate, PEV loop, constrained context graph — is what this post describes. It runs on the substrate you already operate.

The capital case for funding both together, with the evidence appendix and the challenge-mitigation matrix, is the subject of the Investment Blueprint companion to this series.

Series navigation: Part 3 covers KitOps OCI ModelKits as the lifecycle layer for model and skill artifacts. Part 4 covers local-LLM serving on GGUF/NPU and the air-gap licensing constraint. The series hub connects all seven parts under the open-standards frame.

Leadership companion: Who Owns the Read-Only Gate? Governing Agentic Autonomy at the Edge (javatask.systems S3) examines where the human-on-the-loop boundary sits as an organizational design decision, not a compliance afterthought.