Read time: 17 minutes
Agents is one of the most misunderstood terms in the world right now.
I talk with data, tech, and business people, all of whom have different definitions of what the word means and how it applies to AI.
While this is a confusing term, it doesn’t begin to approach the confusion people have about how to embed agents in how you work (and doing so in a scalable, sustainable way).
And because of that confusion, the world is still focused on building agents rather than operationalizing them. Yes, the frontier AI labs and large SaaS vendors are hiring Forward Deployed Engineers like crazy with the goal of operationalizing Agents and AI, but at the forefront, companies and leadership just want to build without thinking about the full rigour of productionization.
Because to operationalize these new AI products, you need to think beyond the simple prompting and consider how DevOps and DataOps principles factor into your builds.
If you think about it, AgenticOps is sort of like the child of both; it pulls from each discipline to ensure that AI agents have the right software principles to continuously operate on data that is trusted to spit out the right outputs or deliver the right outcomes. And this is hard, because combining these two already difficult disciplines also inherits every problem software and data ever had.
A couple of things before we dive into this new domain of the Data & AI Ecosystem.
First, AgenticOps is still evolving as a domain. I’m writing based on the principles I know, I’ve tested, and that I’ve seen work in my consultancy and in other data teams. But the technology will evolve, and approaches/ perspectives will change, so keep that in mind!
Second, while the terminology and technology are considered new, most of AgenticOps is not. You are pulling the relevant pieces from DevOps and DataOps fundamentals, re-applying them to a technology that no longer adheres to the old rules and will redefine how we work in the future. So don’t treat this as a brand-new discipline, and take a read of my last article if you haven’t already.
Let’s get into it!
Before defining AgenticOps, let’s lay out some key definitions for the domain:
AI Agent – An AI-enabled software engine that takes a goal, works out the steps to reach it, and then carries those steps out itself. The LLM is the backend that does the reasoning, but the agent is the whole system around it. The difference between an Agent and an AI chatbot is the fact that the agent acts to get something done (whereas a chatbot gives you an answer).
Agentic – The concept of chaining several steps together and making choices along the way within an autonomous system; this is rather than running one hard-coded path. This idea allows for a spectrum of outcomes/ outputs based on the predictive nature of the AI tool and any other inputs, tools, context, etc. within the system.
Tools – The functions an agent is allowed to call to accomplish its task. Think about MCPs, skills, etc., all of which fit into the agentic toolkit to turn natural language text generation into action.
The Harness – The software infrastructure scaffolding around the model that feeds it context, calls the tools, keeps the loop running, and decides when to stop.
When we think of Agentic AI it is important to keep all these things in mind. For example, when an AI tool or agent underperforms, the instinct for most people is to blame the model or rewrite the prompt. In reality, it is usually the surrounding system that failed to live up to the user’s expectations (e.g., they added the wrong context, didn’t have the right tools/ it did something unexpected, the outcome wasn’t defined properly, etc.).
Now that we know what Agents are, let’s add on the Ops part. Similar to how DevOps bridges the development and operations of software, and DataOps concerns the management of data across operational tools or solutions, AgenticOps is emerging around the development and operations of AI agents. And if you think one level further, this idea comes about through inheritance from these previous domains.
AI obviously borrows from both software and data, merging the two in an incredible way. Therefore…
…From DevOps, agents inherit the software problems. An agent is deployed like software, so it needs versioning (prompt, model, memory, etc.), automated testing before anything ships, observability once it’s live, and a way to roll back if a change doesn’t quite work out.
…From DataOps, agents inherit the data problems. An agent runs on your data, so it needs quality checks on what it consumes, contracts with the systems feeding it, lineage to trace where an answer came from, and a way to track memory/ context across conversations.
Oh, and one more thing that AgenticOps has to deal with that wasn’t the case for either of its predecessors: the thing running the whole output is non-deterministic. For reference, a deterministic system is one where the same input produces the same output, every single time. Ideally (minus package updates, broken code and data changes), that is the framework that DevOps and DataOps should operate on.
However, an LLM doesn’t work that way. It evaluates data, predicts what is next, interprets that prediction, and gives you an output. Unfortunately, what you get is a different response to the identical request (especially since our questions may be prompted/ framed just a tad differently each time). Rather than setting up an operational system that does what it’s told, we now have variability built into it that we can’t fully control.
The final twist in AgenticOps is that the operators are no longer confined to the technical domain. This technology is available to everybody, and companies want to take advantage of this. That changes who this discipline is for. The person scoping an agent is now just as likely to be a business stakeholder as an engineer (therefore with no understanding of DevOps/ DataOps), and most of the decisions that determine whether an agent works are made without the first principles mindset good engineers lead with.
So the short definition: AgenticOps is DevOps plus DataOps, run by a non-deterministic technology in the middle of your system and operated by anybody in the organization.
DevOps and DataOps came about from broken processes in very popular and necessary domains/ industries. AgenticOps will likely emerge in a similar way.
When the vibe coding era started in 2025, it unleashed a whole new way to approach code. AI allowed people who have never built before to create working tools, automations, and entire systems with intent rather than practiced technical skills. Honestly, the thrill of describing something and watching it exist twenty minutes later is pretty awesome, so it’s hard to fault people when they built for speed instead of best practice or scalability.
This type of dev work doesn’t scale. And it’s kind of a throwback to both DevOps, where the development and operations of the software is done in a misaligned manner, and DataOps, where the data quality is not verified in a way that allows for constant usage.
With Agents, instead of the process breaking down due to different people/ teams or a lack of tools, it is competing goals/ intent that impede its operations. Think about it, the initial intent by the AI coding agent is to build the tool to exist, not to be maintained!
In addition to the goals and intent, the thing to add on top of that is the process. We now work at such a pace and scale that, to be productive in this AI world, you have to take shortcuts. If you’re smart, you set up those shortcuts to work and be scalable, but most people haven’t put that foundational work in (especially non-technical folks).
Therefore, you get people building demo-ready pilot products that look great and work for the first while (i.e., the vibe coding produces results). But then you connect it to the real data, shifting context, or new technologies, and the demo fails to scale (or scales in a completely unsustainable way). I mean I even did this, getting excited and vibe coding a knowledge graph without using first principles; unsurprisingly it failed and set me back a week or two.
As we hand more and more of our thinking over to AI, what used to be well-thought-out, principles-first approaches to software and tooling development become generic slop. People are taking AI’s recommendations to all code, meaning they build in the same way as others. Therefore, software or AI tools aren’t differentiated against the competition.
I love POCs and pilot projects. But they often fail. Not because the model is dumb, but because nobody built the operational machinery around it.
So I wanted to explore how to build properly in this new type of world.
There are multiple components to building an agent that we have to learn from. As I wrote about last week, the DevOps and DataOps principles go into it (e.g., proper testing, observability, modular builds, etc.).
But the difference is that these two disciplines are about managing software and data artifacts. AgenticOps manages behaviour. The software and data components still apply, but now we have to think about how these agents act in their environment. This increases the complexity significantly, as you have to consider whether they are acting in the best interest of a spectrum of potential outputs, rather than one defined output. And honestly, what I’m writing today may not stand the test of time and may shift in months/ years to come.
This is the conversation I’m having most often now with clients and customers. They think they can do everything from their AI harness. But should they be doing that? Or should it be a deterministic script completing the task?
The first design decision. If a task has stable logic and explicit criteria (e.g., a calculation, a data pull, a format conversion, a defined rule), it should be a tested and versioned script. This is faster, nearly free, auditable, and performs the same way every time. On the other hand, if you have situations that need nuance and thrive in ambiguity (e.g., interpreting vague requests, synthesizing messy sources, drafting language, making a judgment call), then that is where the LLM/ AI tool comes into play.
Even if you have deterministic tools/ scripts, an AI agent still plays a role in the architecture. For one, it is probably helping build this deterministic solution. For another, the agent is the layer sitting across both. With the right prompt/ context, it can reason about what’s needed and when, then call deterministic tools to do the precise work. Anthropic’s own guidance mimics this, as they no doubt leverage AI when building, but use predictable, predefined workflows over autonomous agents for most well-defined tasks. This is also why you see the best engineering thought leaders and AI practitioners having a contained vault for their coding scripts and deterministic tools (usually callable by MCP or git).
When you move tasks from the LLM side to the coding environment, you get faster, cheaper and more testable results. Moreover, every task left on the LLM side that shouldn’t be there means your AI may give you confident, unpredictably wrong answers, especially if you haven’t set it up properly with the right context or memory.
Every customer and client I know has run into this, and this hinders their AI usage. But they don’t understand it because they are non-technical operators who see Claude or ChatGPT as magic tools that should work.
In the end, start your AgenticOps journey by architecting for what should and shouldn’t be AI.
Everybody is talking about context, and every data SaaS vendor is now positioning themselves as a tool that manages it. Now I’ve written about context layers in detail before, but when thinking about operationalizing AI agents, it goes beyond just this idea. There are three parts to this:
The Data Model – Companies have ignored data modelling for decades. But now, without human intervention, the data has to be structured so it can be read properly and understood for what it is. Otherwise, the agent will misinterpret the information or not be able to find the correct source. Again this references back to data architecture, DataOps, and data quality from source to consumption.
The Context Layer – Go read the article, but this builds on the modelled data to provide the semantics of your business, the unstructured context, and everything in between. Properly combining the raw, quantitative data with the right business meaning allows the agent to understand the data in the same way the business has agreed to understand it.
Memory - This one is unique to AI harnesses and tooling, but basically references what the harness carries forward about a situation. Think about when your AI tool forgets about your other conversations or even remembers you for what you said 3-4 months ago. Managing memory within your harness and AI agents is still a very young domain, and most people haven’t figured it out yet, but it is essential for operationalizing agents in a meaningful way; otherwise, the agent’s recollection goes stale, and it makes mistakes.
When operationalizing for context within your Agentic systems, bigger is not better. The whole idea of doing this draws back to both DevOps and DataOps principles of building in a modular way, testing for freshness and quality, and creating that looped system that your customers and users agree with and can factually confirm is correct. This whole area of AgenticOps is extremely difficult and will be a huge focus for the next few years if businesses really want to take advantage of AI.
So I totally missed this principle in last week’s piece, and Matthew Mullins called me out on it (which I hugely appreciate because his perspective is very valid and comes with a ton of experience).
In DevOps, infrastructure defined in configuration files you version, reuse and share is essential to running in a scalable and automated way. DataOps has the same idea with pipelines, from point-and-click ETL to code-first, modular transformations (i.e., dbt is the most popular example).
For AgenticOps, this becomes even more important because an agent’s native interface is code. It reads and writes code; that’s how it operates on your systems.
So the prompts, tool definitions, model selections and policies are all versioned artifacts that go through review, testing, staged rollout and rollback. This means the codified change is inspectable, and the deterministic gate is a pull request rather than a prompt (going back to our architecture section above). This is how AWS frames it in their Agentic AI Lens, where “treat agent behaviour as code” is a named design principle. 12-Factor Agents makes the same case from the practitioner side.
The other thing to consider is that the agents themselves need infrastructure. Once you are running dozens or hundreds of them, you are provisioning vector stores, API tokens, model routing logic, sandboxed runtimes and guardrail policies for each one. Therefore, you need IaC baked into your system to programmatically manage the permissions, LLM endpoints, environments and security profiles of each agent.
IaC does a good job of underscoring how AgenticOps inherits from DevOps and DataOps: teams who have practiced these principles can easily transfer that to building agents because they’ve done it already with software infrastructure and pipelines.
It’s funny, as humans operating in a capitalist society, I think we are taught that any type of halt on freedom/ our power is a negative thing. However, it is also regulations and guardrails that have given us some of the best things (e.g., holiday/ vacation, a 5 day work week, healthcare, preventing corruption, etc.)
That might be more of an aside, but it relates to this. Because in the world of data & AI, you need to define what the agent is supposed to do and what it is never supposed to do. Otherwise you won’t be able to contain the mess.
From a productivity and cost-management perspective, my best example is when I tell my agent to parse a Word Doc, and then it takes half my 5-hour tokens to build a script to do that (even though it has a skill/ tool that does it already). Or where it goes beyond where an agent should ask, like sending an email, engaging with customers in the wrong way or making business decisions for you. Finally, there is even what happened a few days ago when OpenAI’s models chained vulnerabilities to escape its research sandbox and break into Hugging Face’s production infrastructure. The AI Agent had a goal, and it broke a lot of rules to achieve that goal, likely because the guardails/ constraints weren’t set correctly. As Simon Willison put it: “If you set them a goal and give them a way to get there, even inadvertently, they will figure it out.”
We aren’t going to make sure the AI asks every time before doing something (it is unsustainable). In fact, we’ve built AI in a certain way that goal-directed persistence is a core feature. Therefore, constraints are what make it safe.
Figuring out how to operationalize with the right constraints baked into the tool will be an essential part of future development, especially as agents get more capable (and that ceiling on capability is rising fast).
Quick advice (by no means comprehensive): scope permissions to the specific task, make the agent modular/ short-lived, require confirmation before it hits certain guardrails (governance tiers is a good method here), and give the agent a defined stopping point. There are good resources out there for common guardrails and open source governance principles.
The last bit of AgenticOps I’m going to mention here is something that I don’t see defined that much. It might be because it’s still fairly new and it’s ever-expanding, or it’s hard to define, but I’m going to term it capabilities.
Capabilities are the MCP connectors, the agent skills and the model type. Together they determine what the agent can actually do.
Setting a tool up with the right capabilities is genuinely an art now. It gives the agent the underpinning knowledge and workflow expertise to do a job more effectively. At the same time, a lot of people are also just over-provisioning; there is such a proliferation of skills, model types and connectors that people bolt things on for the sake of it. This takes away from the agent’s ability because it has too much context instead of the right amount.
The discipline is choosing the smallest set that does the job, and making sure it is built with the right toolset. Again, this pulls from DevOps design principles in decomposing work into specialized, bounded tasks (or agents in this case) with declared scope and explicit limits. In the end, a narrow agent is easier to evaluate, secure and trust than one that can do everything.
Skills are worth pulling out on their own. While MCPs and the model type are both powerful, the skills build on the underlying infrastructure with a packaged set of instructions, telling the agent how you want a particular task done. And because a skill is just files, it lives inside the Agents-as-Code approach above: versioned, reviewed, tested, reusable across agents rather than re-prompted from scratch every time somebody needs the same job done. I will speak about skills more next week in my Solutions Architecture article
In addition to all the components I mentioned above, there is also a long list of things that agents will touch when they are operating. For example, security, data governance, data privacy, technological bandwidth, compute and cost.
All of these things matter and need to be considered when doing AgenticOps, but it is also hard for me to classify them within the same domain. In my opinion, folding them in creates significant scope creep for the term, especially now that agents are a part of everything operational when it comes to technology.
The key thing is to align your approach rather than absorbing it into the domain. Ensure the teams/ individuals who own security, data, infrastructure, and governance are involved in how your agent works and what it’s allowed to touch. And do this from the beginning rather than when something breaks.
Moreover, by not extending the domain significantly, we avoid a generalized role that covers everything. I have a real problem with the engineering role because companies set unrealistic expectations for those resources to do absolutely everything. I can easily see AgenticOps Engineers (which I think right now is classified as the FDE, Forward Deployed Engineer, role) falling in the same category.
While we are still at the beginning stages of AgenticOps and AI agents, this domain is proving to be very fruitful for many teams and organizations. The components and factors in AgenticOps really come down to best-practice engineering principles meeting a genuinely new technology, working out what an agent actually is, and engineering a conducive way to work with it.
Just remember, AI agents aren’t just technology; they are behavioural, intelligent pieces of software, meaning you need to shift how you think, especially when operationalizing them with these best practice engineering principles. So I will leave you with a few high-level pointers to build on the components I defined above:
Continuously Evaluate Behaviour – Testing, observability and monitoring tells you if your agent ran and completed a task, but it doesn’t tell you if the agent succeeded in its task. Build behavioural evals (repeatable test cases to score the quality of an agent’s decisions) into your approach. This ability to evaluate and tweak agents will no doubt grow and improve over time, so keep it in mind!
Roll Out Cautiously – Agentic behaviour will naturally drift, and you can’t fully predict it. So ship it the way you’d ship a risky release and evaluate/ align behaviour against your AgenticOps principles when pushing to production.
Human Accountability – Agentic AI is still a new technology and where an agent’s output feeds a decision, you still need to have a person who can answer for it. And ensure this person understands what the agent is doing and what its goal is. This too is why AgenticOps is not just for technical people, but has to be built into the rest of the organization who is using AI as well.
These three areas build on the fundamentals from last week and the principles I explained above. And because AgenticOps is not a purely technical domain (so much of its success is deciding what the agent should not do, what a good answer looks like, or which definitions are right), this domain needs to be properly understood across the organization.
AgenticOps is young, the tooling is immature, and the domain will no doubt evolve. But the direction is clear enough to act on and most of the things to know are inherited from DevOps and DataOps.
So like anything in the Data & AI Ecosystem, think about this new era in a holistic way and you will be much better off to deliver the ROI you are looking for when it comes to AI Agents.
Next week we will go beyond the Ops portion into a role/ domain that helps you build these Agentic (or non-agentic) solutions at scale—Solutions Architecture. This is going to be a great article about one of the most underrated jobs and domains in the data & AI world. Until then, have a great weekend and see you next Sunday!
Thanks for the read! Comment below and share the newsletter if you think it’s relevant! Feel free to also follow me on Substack, LinkedIn, and Medium, or reach out if you are looking for some top-notch freelance consulting input! See you amazing folks next week!
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.