Building your first agentic system is much different from building software solutions or machine learning models. However, knowing one of the two greatly helps you frame the problem and create a workflow you’ll follow.
A lot of people have this illusions that AI agents are just chatbots or workflows that function on hard rules, and that anyone can build them in 5 minutes. Perhaps, you can, but it won’t work the way you imagined it and it won’t be long before its functionality is fully broken.
In this post, we’ll skip the universal assistant. Instead we’ll build a system that does one specific job, well enough that you can tell it worked. Here’s a full step-by-step build, using an engineering review assistant as the working example (as the vast majority of readers have tech background.)
Attention: The guide is very long. If you are reading this from email. Make sure to open the full version.
Do not start with “build an agent that helps me code.” Start with something you can finish a sentence about.
Build an agent that reviews pull requests for design and security issues before a human reviewer looks at them.
For that one job, write down five things.
What goes in: a diff, the PR description, the affected files
What comes out: a structured review with flagged issues and suggested fixes
What context it needs: the team’s style guide, past review comments, the service’s architecture notes
What tools it needs: diff parser, style checker, security pattern scanner
What should never happen without you: approving the PR, merging, commenting publicly on someone’s code without a human reading it first
Job: Engineering Review Assistant
Input: PR diff, PR description, changed file list
Output: structured review with severity-tagged findings
Context needed: style guide, prior review history, architecture notes
Tools: parse_diff, check_style, scan_for_risks
Requires approval: posting the review, approving the PR
Skipping this step is why people end up with a reviewer that flags fifty things nobody asked about and misses the one that mattered.
The narrower the job, the easier the system is to design, evaluate, and improve.
Before your AI agent can reason, it needs to decide what kind of review your task actually needs. That’s the first working piece.
Write down how a human would complete the task.
For example:
Trigger → Collect sources → Filter irrelevant information → Rank important items → Create summary → Verify claims → Draft output → Request approval
This matters because an agentic system is not just a prompt.
It is a workflow containing decisions, state transitions, tool calls, failure conditions, and stopping rules.
Before writing code, identify:
deterministic steps
decisions requiring reasoning
external data requirements
actions with real-world consequences
points where the system should stop
Not every step needs an agent. In many cases, the best architecture combines deterministic automation with agentic reasoning.
That is the whole router. It does not need to be clever. It needs to correctly separate a routine formatting PR from a schema migration touching production data.
The planner turns “review this PR” into a visible list of steps, so you can see what the agent intends to check before it checks it.
At the center of the system is a loop:
Observe → Reason → Act → Evaluate
Nothing here is model-generated yet. It is a lookup table, which means when the review misses something, you know exactly which checklist to fix, not which prompt to reword.
A single dump of “every past PR ever” makes the reviewer worse, because it has to sift through irrelevant history to find what matters for this diff. Split it by purpose.
Working memory: the current diff, description, and file list Standing preferences: the team’s style guide, naming conventions, things that do not change PR to PR History: past review comments and whether they were accepted or overridden Reference material: architecture docs, service ownership maps
Tracking whether past comments were accepted or overridden matters more than people expect. A reviewer that keeps flagging things engineers keep dismissing is training itself to be ignored.
A tool should have one job and a name that says exactly what it checks.
A model cannot read your repo, run a linter, or check who owns a service. It only knows what is in the text you send it. A tool is how you close that gap. It is a plain function with three parts: a name that says exactly what it does, an input the model can fill in, and an output the model can read back and reason about.
That is the entire definition. Nothing about “tool” implies intelligence. The intelligence is the model deciding when to call it. The tool itself should be the most boring, predictable piece of the whole system.
For an engineering review agent, it helps to think about tools in layers, based on what kind of question they answer.
Inspection tools, which read the change itself
Static analysis tools, which check the change against fixed rules
This is where most first attempts stop. It is also where a review agent is still just a fancier linter. The more useful layer is the next one.
Context tools, which compare the change against things the model cannot see in the diff alone
These three are the ones that make the agent feel like an actual senior reviewer instead of a linter with a chat interface. A linter can tell you a function is missing a docstring. It cannot tell you this exact file caused an outage eight months ago, or that three other services quietly depend on the function you just renamed. That kind of check only exists if you build a tool that goes looking for it.
An engineering review agent needs clear rules, written down, not implied by a prompt.
Example system policies:
1. Never post a review comment without human approval.
2. Never approve or merge a PR automatically.
3. Never request changes on someone's behalf without review.
4. Always flag when a changed file has no matching test update.
5. Always disclose when a finding is a guess versus a confirmed match.
6. Ask for confirmation before contacting another team about their service.
7. Log every tool call and every finding it produced.
8. Stop and flag for a second reviewer after a fixed complexity threshold.
Reading a diff and generating comments is cheap. Posting to a real PR or approving it is not.
This one function is why an engineering review agent is safe to run on every PR instead of just the low-stakes ones. It can read and draft freely. It cannot post or approve without a human in the loop.
Agentic systems should not treat their own outputs as automatically correct.
Add verification before important actions.
Depending on the workflow, verification may include:
checking whether sources actually support a claim
validating structured outputs
confirming required fields
comparing results against explicit rules
running a second evaluation step
requesting human approval
The verifier checks the draft review against a short list of concrete, checkable things, not a subjective “is this a good review.”
This will not catch a subtly wrong architectural judgment. It will catch a review that flagged the same issue three times, or one that produced nothing at all on a fifty-line diff, before a human wastes time reading it.
This is where trust gets built, not through a confident-sounding review, but through knowing exactly when the system waits for you.
Low risk, runs alone: parsing the diff, running style checks, drafting findings Medium risk, drafts and waits: a full structured review comment High risk, always waits: approving the PR, requesting changes, merging Never automatic: overriding a human reviewer’s decision
The requires_approval set from step 6 is already enforcing this. The discipline is making sure every code path actually checks it, including the ones you added later and forgot were in scope.
Once a review runs through routing, planning, tool calls, and verification, you need a record of what happened at each stage, or a wrong review becomes a mystery instead of a fixable bug.
A run log for one PR review might look like this.
Without this, "the agent flagged something wrong on PR 482" stays unsolvable. With it, you can see exactly which check produced the bad finding.
Log:
model decisions
tool calls
failures
retries
verification results
human overrides
task completion
Do not start with a multi-agent architecture. The best system is often the simplest one.
Build the smallest version that completes one useful workflow.
Then observe what actually happens.
Your first version should help you answer:
Where does the system fail?
Which steps are unnecessary?
Where does it need more context?
Where is deterministic logic better than reasoning?
Which decisions require human judgment?
Only after answering those questions should you add more tools, more memory, more autonomy, or more agents.
Now let’s put every piece from steps 2 through 9 into one system.
This version can:
Accept a PR description, changed file list, and raw diff
Route it to the right kind of review
Build a plan of what to check
Run every tool from step 5, inspection, static analysis, and context tools
Verify the findings before anything goes further
Check whether posting requires human approval
Log every step along the way
Example memory, fully populated this time, not just a placeholder list:
python
Run it:
python
That produces something like this.
Notice what happened without a single line of model-generated text. The agent caught a hardcoded credential, a bare except, flagged that two other services depend on the file being touched, and surfaced a related incident from March, all before deciding it still needs a human to actually post any of it.
This is intentionally rule-based rather than model-generated, so you can see the full mechanics with nothing hidden. But even this version is already a real agentic system, because it has:
A router that decides what kind of review this is A plan you can inspect before anything runs Layered memory instead of one undifferentiated pile Tools with one job each, including ones that see context a linter never could Risk tiers that block posting and approving by default A verifier catching duplicate or missing findings A full log of every decision, in order
Swap run_tools and generate_output style logic for real model calls where judgment is genuinely needed, keep everything else exactly as it is, and this becomes the review agent you actually run on real PRs.
None of these ten pieces are impressive alone. A router. A checklist. Layered memory that tracks what got overridden. Narrow tools with one job each. A risk list. A verifier that catches duplicates. A log. Put together, they are the difference between a review bot people learn to ignore and one they actually trust, because when it flags something wrong, you know exactly which of the ten pieces to fix.
That is the actual shape of an agentic system. Not one clever prompt. Ten small, boring, inspectable pieces working together.
What’s next:
Replacing the rule-based checks with real reasoning, the version that reads a diff the way a senior engineer would and explains why something is risky, not just that it matched a pattern
A second build many of you have asked for, a research agent that searches, reads, cross-checks itself, and hands you a summary you can actually trust
If you want both builds, full code, no simplified version:
Subscribe to my Substack to be the first one to get exclusive tips, career guides and projects with code + 3 exclusive posts per month.
If you want the complete system to develop agentic AI systems, orchestrate agents and become AI-savvy:
Get Agentic Intelligence Guide
Agentic Intelligence is the full guide, the complete architecture, the reasoning layer, multi-agent coordination, and the production concerns one blog post cannot cover
You will get 3 free guides on OpenClaw, n8n and Claude Code
If the ten steps above made sense to you, the rest of it will move fast
Check my other Guides & Resources
Until next Friday,
Danica
No posts

Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.