RSS Amplifier

Danica's Substack | Data Science, AI & Career · Jul 10, 2026

How to build your first Agentic System (For Personal Use)

0
Sign in to vote or save

Danica Simic · Danica's Substack | Data Science, AI & Career

Building your first agentic system is much different from building software solutions or machine learning models. However, knowing one of the two greatly helps you frame the problem and create a workflow you’ll follow.

A lot of people have this illusions that AI agents are just chatbots or workflows that function on hard rules, and that anyone can build them in 5 minutes. Perhaps, you can, but it won’t work the way you imagined it and it won’t be long before its functionality is fully broken.

In this post, we’ll skip the universal assistant. Instead we’ll build a system that does one specific job, well enough that you can tell it worked. Here’s a full step-by-step build, using an engineering review assistant as the working example (as the vast majority of readers have tech background.)

Attention: The guide is very long. If you are reading this from email. Make sure to open the full version.

Do not start with “build an agent that helps me code.” Start with something you can finish a sentence about.

Build an agent that reviews pull requests for design and security issues before a human reviewer looks at them.

For that one job, write down five things.

  1. What goes in: a diff, the PR description, the affected files

  2. What comes out: a structured review with flagged issues and suggested fixes

  3. What context it needs: the team’s style guide, past review comments, the service’s architecture notes

  4. What tools it needs: diff parser, style checker, security pattern scanner

  5. What should never happen without you: approving the PR, merging, commenting publicly on someone’s code without a human reading it first

Job: Engineering Review Assistant

Input: PR diff, PR description, changed file list

Output: structured review with severity-tagged findings

Context needed: style guide, prior review history, architecture notes

Tools: parse_diff, check_style, scan_for_risks

Requires approval: posting the review, approving the PR

Skipping this step is why people end up with a reviewer that flags fifty things nobody asked about and misses the one that mattered.

The narrower the job, the easier the system is to design, evaluate, and improve.

Before your AI agent can reason, it needs to decide what kind of review your task actually needs. That’s the first working piece.

Write down how a human would complete the task.

For example:

Trigger → Collect sources → Filter irrelevant information → Rank important items → Create summary → Verify claims → Draft output → Request approval

This matters because an agentic system is not just a prompt.

It is a workflow containing decisions, state transitions, tool calls, failure conditions, and stopping rules.

Before writing code, identify:

  • deterministic steps

  • decisions requiring reasoning

  • external data requirements

  • actions with real-world consequences

  • points where the system should stop

Not every step needs an agent. In many cases, the best architecture combines deterministic automation with agentic reasoning.

That is the whole router. It does not need to be clever. It needs to correctly separate a routine formatting PR from a schema migration touching production data.

The planner turns “review this PR” into a visible list of steps, so you can see what the agent intends to check before it checks it.

At the center of the system is a loop:

Observe → Reason → Act → Evaluate

Nothing here is model-generated yet. It is a lookup table, which means when the review misses something, you know exactly which checklist to fix, not which prompt to reword.

A single dump of “every past PR ever” makes the reviewer worse, because it has to sift through irrelevant history to find what matters for this diff. Split it by purpose.

Working memory: the current diff, description, and file list Standing preferences: the team’s style guide, naming conventions, things that do not change PR to PR History: past review comments and whether they were accepted or overridden Reference material: architecture docs, service ownership maps

Tracking whether past comments were accepted or overridden matters more than people expect. A reviewer that keeps flagging things engineers keep dismissing is training itself to be ignored.

A tool should have one job and a name that says exactly what it checks.

A model cannot read your repo, run a linter, or check who owns a service. It only knows what is in the text you send it. A tool is how you close that gap. It is a plain function with three parts: a name that says exactly what it does, an input the model can fill in, and an output the model can read back and reason about.

That is the entire definition. Nothing about “tool” implies intelligence. The intelligence is the model deciding when to call it. The tool itself should be the most boring, predictable piece of the whole system.

For an engineering review agent, it helps to think about tools in layers, based on what kind of question they answer.

Inspection tools, which read the change itself

Static analysis tools, which check the change against fixed rules

This is where most first attempts stop. It is also where a review agent is still just a fancier linter. The more useful layer is the next one.

Context tools, which compare the change against things the model cannot see in the diff alone

These three are the ones that make the agent feel like an actual senior reviewer instead of a linter with a chat interface. A linter can tell you a function is missing a docstring. It cannot tell you this exact file caused an outage eight months ago, or that three other services quietly depend on the function you just renamed. That kind of check only exists if you build a tool that goes looking for it.

An engineering review agent needs clear rules, written down, not implied by a prompt.

Example system policies:

1. Never post a review comment without human approval.

2. Never approve or merge a PR automatically.

3. Never request changes on someone's behalf without review.

4. Always flag when a changed file has no matching test update.

5. Always disclose when a finding is a guess versus a confirmed match.

6. Ask for confirmation before contacting another team about their service.

7. Log every tool call and every finding it produced.

8. Stop and flag for a second reviewer after a fixed complexity threshold.

Reading a diff and generating comments is cheap. Posting to a real PR or approving it is not.

This one function is why an engineering review agent is safe to run on every PR instead of just the low-stakes ones. It can read and draft freely. It cannot post or approve without a human in the loop.

Agentic systems should not treat their own outputs as automatically correct.

Add verification before important actions.

Depending on the workflow, verification may include:

  • checking whether sources actually support a claim

  • validating structured outputs

  • confirming required fields

  • comparing results against explicit rules

  • running a second evaluation step

  • requesting human approval

The verifier checks the draft review against a short list of concrete, checkable things, not a subjective “is this a good review.”

This will not catch a subtly wrong architectural judgment. It will catch a review that flagged the same issue three times, or one that produced nothing at all on a fifty-line diff, before a human wastes time reading it.

This is where trust gets built, not through a confident-sounding review, but through knowing exactly when the system waits for you.

Low risk, runs alone: parsing the diff, running style checks, drafting findings Medium risk, drafts and waits: a full structured review comment High risk, always waits: approving the PR, requesting changes, merging Never automatic: overriding a human reviewer’s decision

The requires_approval set from step 6 is already enforcing this. The discipline is making sure every code path actually checks it, including the ones you added later and forgot were in scope.

Once a review runs through routing, planning, tool calls, and verification, you need a record of what happened at each stage, or a wrong review becomes a mystery instead of a fixable bug.

A run log for one PR review might look like this.

Without this, "the agent flagged something wrong on PR 482" stays unsolvable. With it, you can see exactly which check produced the bad finding.

Log:

  • model decisions

  • tool calls

  • failures

  • retries

  • verification results

  • human overrides

  • task completion

Do not start with a multi-agent architecture. The best system is often the simplest one.

Build the smallest version that completes one useful workflow.

Then observe what actually happens.

Your first version should help you answer:

  • Where does the system fail?

  • Which steps are unnecessary?

  • Where does it need more context?

  • Where is deterministic logic better than reasoning?

  • Which decisions require human judgment?

Only after answering those questions should you add more tools, more memory, more autonomy, or more agents.

Now let’s put every piece from steps 2 through 9 into one system.

This version can:

  1. Accept a PR description, changed file list, and raw diff

  2. Route it to the right kind of review

  3. Build a plan of what to check

  4. Run every tool from step 5, inspection, static analysis, and context tools

  5. Verify the findings before anything goes further

  6. Check whether posting requires human approval

  7. Log every step along the way

Example memory, fully populated this time, not just a placeholder list:

python

Run it:

python

That produces something like this.

Notice what happened without a single line of model-generated text. The agent caught a hardcoded credential, a bare except, flagged that two other services depend on the file being touched, and surfaced a related incident from March, all before deciding it still needs a human to actually post any of it.

This is intentionally rule-based rather than model-generated, so you can see the full mechanics with nothing hidden. But even this version is already a real agentic system, because it has:

A router that decides what kind of review this is A plan you can inspect before anything runs Layered memory instead of one undifferentiated pile Tools with one job each, including ones that see context a linter never could Risk tiers that block posting and approving by default A verifier catching duplicate or missing findings A full log of every decision, in order

Swap run_tools and generate_output style logic for real model calls where judgment is genuinely needed, keep everything else exactly as it is, and this becomes the review agent you actually run on real PRs.

None of these ten pieces are impressive alone. A router. A checklist. Layered memory that tracks what got overridden. Narrow tools with one job each. A risk list. A verifier that catches duplicates. A log. Put together, they are the difference between a review bot people learn to ignore and one they actually trust, because when it flags something wrong, you know exactly which of the ten pieces to fix.

That is the actual shape of an agentic system. Not one clever prompt. Ten small, boring, inspectable pieces working together.

What’s next:

  • Replacing the rule-based checks with real reasoning, the version that reads a diff the way a senior engineer would and explains why something is risky, not just that it matched a pattern

  • A second build many of you have asked for, a research agent that searches, reads, cross-checks itself, and hands you a summary you can actually trust

If you want both builds, full code, no simplified version:

  • Subscribe to my Substack to be the first one to get exclusive tips, career guides and projects with code + 3 exclusive posts per month.

If you want the complete system to develop agentic AI systems, orchestrate agents and become AI-savvy:

Get Agentic Intelligence Guide

  • Agentic Intelligence is the full guide, the complete architecture, the reasoning layer, multi-agent coordination, and the production concerns one blog post cannot cover

  • You will get 3 free guides on OpenClaw, n8n and Claude Code

  • If the ten steps above made sense to you, the rest of it will move fast

  • Check my other Guides & Resources

Until next Friday,
Danica

No posts

Read the original on danicasimic.substack.com

Comments

Nothing yet. Say the first thing.

    Sign in to join the conversation.