Your agent said everything looks fine. Scroll up.

Hi there!

Today’s story is about an agent that told an engineer a device was fine while its own tool output said otherwise. Before we get into it, Sif Baksh, whose book this issue draws from, is running a hands-on workshop on August 25th that walks through exactly this problem: building the evidence checks that catch an agent lying to you in fluent prose.

Claim 40% Off

If you’ve ever had an agent tell you everything looked fine while the raw output said otherwise, you already know why this session exists. In this four-hour hands-on workshop, you build a RAG-powered NetOps assistant that answers from your own runbooks and device configs, then build the evidence record and response checks that stop it from writing a confident summary its own tool output disagrees with. Same failure mode as today’s story, except you’re the one building the guardrail.

Four hours. Four labs. For AI Network Engineer readers, use code LIMITED40 for 40% off all passes.

P.S. My book, Building AI Agents for Network Operations, goes deeper into the same architecture. You can bundle it with your event pass at checkout, or pick it up separately if the 25th does not work for your calendar.

That's the hands-on version. Now, the story.

Your troubleshooting agent finishes its run and gives you this:

Clear. Confident. Well-written.

Now scroll up to the tool output it was summarizing. total_peers: 2. established_peers: 1. Neighbor 10.1.2.2 sitting in Idle with zero prefixes received. And Ethernet3, described as server_rack_2, is down.

The prose is fluent. The evidence disagrees.

This is one of the easiest failure modes to miss in agentic troubleshooting, especially in a polished demo.

A tool-using agent produces two distinct things.

• First, the tool results: what get_bgp_summary() actually returned and what ping_device() actually reported.

• Second, the final summary: natural language written about those results.

The second is another model-generated output. It can omit, soften, or contradict facts that appeared in the tool results, even when the underlying investigation was correct.

During an incident, a confident wrong summary is worse than no summary. It sends someone down the wrong path with the reassurance of a second opinion attached.

So the rule is simple: when the summary and the tool output disagree, the evidence wins. Always.

The final model response is never more authoritative than the tool results it describes.

That means you need something in your system that can tell when they disagree.

You can reduce the drift with prompt rules, and you should.

Instructions such as always compare established_peers against total_peers, always name any neighbor not in Established state, treat zero prefixes received as possible routing impact, and never claim there are no issues if a tool returned an error or degraded state can meaningfully improve the output.

But prompt rules are guidance, not enforcement.

They make the good answer more likely. They don’t make the bad answer impossible. If your only defence against a wrong conclusion is a bullet point in a system prompt, you don’t have a defence.

The enforcement has to live outside the model.

Sif Baksh demonstrates a useful approach in Building AI Agents for Network Operations.

Alongside the conversation history the model reasons over, the application builds a separate, structured record of what was checked and what looked unhealthy. The model doesn’t write this record. Your code assembles it from the raw tool results.

For the leaf2 case, the record holds:

• Device status: up

• Down interface: Ethernet3, described as server_rack_2

• Total BGP peers: 2

• Established BGP peers: 1

• Non-established neighbor: 10.1.2.2, Idle, with zero prefixes received

Once those facts exist as data rather than prose, your application can derive rules from them:

• If established_peers < total_peers, the response must flag a BGP issue.

• If a neighbor isn’t Established, the response must identify that neighbor.

• If a neighbor reports zero prefixes received, the response should flag possible route impact.

• If ping_device() returns unreachable, the response must not describe the target as reachable.

• If a tool returns an error, the response must surface it or request another check.

The evidence checks themselves do not need another model. Your code can derive them directly from the tool results.

To enforce them reliably, require the final response in structured fields before rendering it as prose. That gives the application something deterministic to validate instead of searching an unrestricted paragraph for the right conclusion.

Say a host at 10.99.99.99 is unreachable and the agent is asked whether BGP on leaf2 might be involved.

The ping tool returns 100% loss. That proves reachability failed. It does not prove why.

If the BGP summary also shows an Idle neighbor, the honest answer is that the failed session may be related to a missing route, not that BGP caused it.

Hedging in a troubleshooting summary can look like weakness. Here, it’s the opposite.

An agent that separates what it confirmed from what it inferred is far more useful during a 2 a.m. incident.

Give your agent’s final response a fixed shape with five structured parts:

• Finding: The short operational conclusion.

• Evidence: The specific tool results supporting it, recorded as facts. For example: leaf2 status is up, BGP total_peers is 2, BGP established_peers is 1, and neighbor 10.1.2.2 is Idle.

• Likely cause: A clearly labelled hypothesis, using may be or likely rather than certainty the evidence doesn’t support.

• Next checks: What the engineer should verify or approve.

• Unknowns: What the agent did not check.

That last section is the one to fight for. It keeps the agent honest.

If the route table was never inspected directly, the response should say so. If only reachability was tested, it should say that the route itself wasn’t examined.

Then run your evidence checks against those structured fields before displaying the final answer. If a check fails, don’t show the answer. Show the mismatch.

The summary is a view of the evidence. It is not the evidence itself.

When the two disagree, trust what the tools returned; not how confidently the agent described it.

Thanks,

Sayali

Editor-in-Chief

This issue draws on Building AI Agents for Network Operations by Sif Baksh. Chapter 7 follows the leaf2 investigation step by step, including the evidence record and response checks.

Get the Book

The evidence-versus-summary problem doesn't stop at troubleshooting. It shows up the moment an agent proposes a change, or the moment you're deciding who's allowed to approve one. That's the thread running through the next two sessions.

Aug 27th · Agentic AI for Infrastructure Engineering: From Chatbots to Operators

Ritesh Vajariya, founder & CEO - AI Guru, teaches you how to build an infrastructure agent that reads pod events, correlates live metrics, and proposes fixes it only executes once you approve, replacing the documentation chatbot most platform teams already outgrew. Seven production-grade failure scenarios, injected into your own local Kubernetes cluster, yours to keep and re-run after the session ends.

Book Now

If your Docker and Kubernetes fundamentals need shoring up before you get there, The Ultimate Docker Container Book (4th edition) by Dr. Gabriel Schenker is the deepest single resource we carry on containers through orchestration, and its latest edition adds AI-driven DevOps patterns on top.

Check it out on Amazon

Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture

Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.

Book Now

Since the session is already an add-on to the Azure basics going in, Microsoft Azure Fundamentals Certification and Beyond (built around the January 2026 AZ-900 update) is worth having on hand beforehand, especially if you’re eyeing the certification alongside the hands-on identity work.

Check it out on Amazon

Before you go, I’d like to hear from regular CloudPro readers. Hit reply and tell me what would be most useful for your team. And if this issue helped sharpen the conversation, please like it and leave a comment so more platform, cloud, security, and infrastructure folks can find it.