Author: BradMoon
Over the past few years, nearly every possible path for applying large models to code auditing, vulnerability discovery, and security analysis has been tried: static analysis, dynamic analysis, symbolic execution, RAG, workflow orchestration, code slicing, long pipeline frameworks, and all kinds of academic prototypes. Each of these approaches played a role at different points, and together they pushed the field to where it is today.
But if you place all of those explorations back on the same timeline, one thing becomes increasingly clear: the way AI security tools are built has reached a real turning point.
That turning point is not just about stronger models, more tools, or more people entering the field. The deeper shift is that the cognitive framework of the field itself is changing.
If you keep unfolding that timeline, a fairly clear evolutionary line appears:
AI for Security is moving from framework-driven to Coding-Agent-driven; from hand-building complex flows to first reproducing the human process and then automating it; from manually pre-adjudicating information to letting agents explore richer material more autonomously.
This article mainly tries to answer three questions: how AI code auditing evolved over the past few years, why the old heavy-orchestration and heavy-preprocessing approach is no longer a good default, and what the right starting point is if you were building your own AI security tooling today.
Part of why I wanted to write this article is that, over the past few months at@CertiK , the internal AI Scan system we have been building is now getting close to release. That scanner has taken up most of my attention, energy, and thought for quite a while, so writing this piece is also, in a very personal sense, a way to pause, summarize the path, and give myself an honest account of what this period of work has really meant. I will probably say more about that in Part II.
The views and opinions expressed in this article are solely my own and do not necessarily reflect those of CertiK.
If you roughly stretch the timeline from late 2022 to early 2026, the evolution of this field can be divided into two broad phases.
The first phase was dominated by academic exploration while engineering was still in its early stage. This phase lasted roughly until before January 2025, and can itself be split into three smaller moments.
At the beginning, the field was primarily static-analysis-first. The earliest wave of work was, in essence, about getting large models involved in static code analysis. The key ideas of that period were SAST, using large models to interpret business logic, and using LLMs to help identify vulnerability context. Put simply, the core idea was that traditional static analysis would throw code structure at the model, and the model would handle business understanding. Academically, that mattered a lot because it proved for the first time that LLMs were not just useful for code generation. They could also participate in program understanding, security reasoning, and vulnerability discovery. But the limitation was obvious: most of it was still paper-style validation, far from something that could really be used as an engineering product.
Later, the field moved from static toward dynamic methods. Research expanded into more complex approaches, including dynamic analysis, symbolic execution, and tighter integration with traditional program analysis tools. The defining realization of that stage was that static analysis plus a large model was not enough. More runtime, path-level, and constraint-level information had to be brought into the loop. This was also when a number of representative works began to appear, trying from different angles to combine LLMs with traditional security analysis capabilities and push the methodology forward.
By the second half of 2024 and around January 2025, the field entered a relatively mature stage. Maturity did not mean the problem was solved. It meant the consensus had become clearer: models alone were not enough, and traditional tools alone were not enough either. The more realistic path was to combine them more tightly. At this stage, many systems began to take on clearer engineering forms. They were still complex, but they were no longer just paper demos. They were starting to look like real systems.
If I had to summarize this first phase in one sentence, it would be this: from primarily static, to dynamic augmentation, to tool-model integration.
Once you understand that history, the changes after 2025 become much easier to interpret. Because what changed was not simply that there were more tools or more participants, but that the methodology itself shifted gears.
If the previous phase was still mainly academic exploration and early engineering, then starting in 2025, especially from late 2025 into early 2026, the field entered a very different stage: engineering exploded.
The most visible change was that more and more teams were able to build their own “large model + security” systems. In other words, this stopped being something only a few research teams could experiment with. It became an engineering direction that could be broadly attempted, rapidly iterated, and continuously shipped.
But if you only interpret that as “more people started doing it,” that still misses the real point. The actual turning point was that the methodology began to shift collectively.
Previously, many systems were organized around workflows and frameworks: using things like LangChain and LangGraph to build flows, manually preprocessing code and documents, manually managing RAG, manually wiring up SAST, AST, dynamic tools, and validation, and then encoding the whole reasoning process into a fixed pipeline.
That made sense at the time, because model capability was weaker and had to be carefully hand-held.
But once the new generation of Coding Agents appeared, the situation started to change. With tools like Claude Code, Codex, and Cursor, many people realized almost at the same moment that the information filtering, tool selection, task decomposition, and flow stitching that developers used to hard-code by hand could increasingly be handed over to the agent itself.
That is why 2026 feels like such a visible milestone. From that point on, the mainstream idea increasingly became: stop hard-coding the entire process with heavy frameworks; stop predefining every workflow step; stop obsessing over manually maintaining large amounts of preprocessing and prompt assembly logic. Instead, give the agent a clear goal, the necessary tools, and some constraints, and let it decide what to read, what to use, and how to proceed, then let it reflect and iterate on the results.
In other words, the field is moving from framework-driven to Coding-Agent-driven.
There is also a sharper and more realistic judgment worth adding here. Before 2026, many companies and people, whether because of path dependence, organizational inertia, or simply lack of understanding, spent years underestimating LLMs and sometimes even dismissing them outright. Then once the direction became obvious, they suddenly wanted to stage a great leap forward, as if the entire capability stack could be caught up overnight. That shift is absurd in itself.
Because what really determines whether a company, a team, or an individual can build something useful has never been how loudly they shout today. It has always been whether they have been accumulating understanding over time. Do they really understand the tendencies, probabilities, and instability boundaries of LLMs? Do they understand when models drift, when they hallucinate, when they should be trusted, and when they must be constrained? And have they actually baked that understanding into products through repeated iteration? Without that accumulation, concentrated effort later on often produces things that look new, but are not genuinely useful.
That is exactly why understanding why this paradigm shift happened matters much more than merely repeating slogans about it. So the next question is obvious: why is the traditional framework-driven route becoming a bad default?
One judgment can be stated very directly: it is not that traditional frameworks are completely useless, but that they are no longer a good default starting point for building AI security tools today. The problem is not whether they have value. The problem is that they are increasingly ill-suited as the starting point for a new generation of systems.
The first reason is that the engineering is too heavy. Developers have to solve everything themselves: how to filter material, how to slice code, how to build RAG, how to analyze call relationships, how to insert static analysis results into the loop, how to feed those results back to the model, how to do planning, and how to do validation. Every link in the chain has to be designed, stitched together, and tuned by hand. The end result is usually the same: a huge codebase, a complex workflow, and a system where changing one thing affects everything else.
The second reason is that manual preprocessing is extremely expensive. In many old systems, the heaviest part of the pipeline was essentially deciding in advance what the agent should see and what it should not. To provide the model with a human-predefined “just right” amount of material, teams did a lot of preprocessing: retrieval, slicing, aggregation, extracting call chains, trimming path information, and so on. Those approaches were indeed necessary when model capability was weaker and the amount of readable material was more constrained. But they came with two problems. First, they were complex to engineer. Second, they often made the wrong information decision too early and dropped the key detail before the agent ever got a chance to see it. Many times, what looked like helping the agent understand the code was actually making the wrong filtering decision on the agent’s behalf.
The third reason is that once a workflow is hard-coded, it stops evolving. If you turn the whole system into a fixed flow, for example static analysis, then slicing, then one round of pre-filtering, then the model, then validation, it may look neat. But the real problem is that it becomes frozen. Once the result is bad, it becomes very hard to know what to change. Was it the prompt, retrieval, slicing, planning, or tool ordering? And once model capability changes, the whole fixed flow may become outdated at once.
So the problem with framework-driven systems is not just that they are cumbersome. The deeper problem is that they are not a natural fit for an era in which models are changing rapidly.
Once the problems of the old route are clear, the next real question is obvious: if frameworks are no longer the right starting point, what is?
If I had to compress the whole methodology into one sentence, it would be this: the essence of an agent is reproducing the human process.
That sentence sounds simple, but it can almost serve as a general principle for building AI security tools today. If the previous section explained why the old method is increasingly unsuitable, this section asks where the new methodology should actually begin.
One very common misunderstanding is to start by asking how the system should be built. I actually raised this point as early as an internal company discussion in mid-2025. Unfortunately, not many people really listened at the time. When many people receive a task, their first reaction is still something like: should I use LangChain, should I plug in RAG first, should I do code slicing, should the workflow be split into multiple layers? But that is the wrong first question.
The thing you actually need to figure out first is not what framework to use. It is how a human would do this task, which steps are essential, what information must be obtained, what experience determines result quality, and how the human and the agent should cooperate most smoothly. Only after those questions are clear does automation have a real foundation.
The practical starting point is to let humans and coding agents run the task together first. Do not try to build the final system in one shot. Let a human and the agent complete the task together first.
For example, in root cause analysis, you can begin by pulling the necessary data, then have a human work together with Cursor or Codex, compare the result against ground truth, and if the result is wrong, add more information, adjust the prompt, and change the collaboration pattern until the process works.
The same applies to white-box code auditing. Start by writing an operations guide and let the agent follow it for the first phase, such as breaking down the architecture and building an initial understanding. Then add a more detailed guide and let it continue deeper into vulnerability discovery. The human only intervenes at key points. Once the process becomes stable enough, the human can gradually be removed.
The key to this methodology is not saving effort early. It is discovering the real execution path of the task first.
Once “human + agent” can reliably get the task done, automation becomes an engineering problem. In other words, automation should not be the first step. It should be the last step.
Once you follow the logic above, you naturally arrive at a core question: how should human experience actually be accumulated? That is where an important concept appears: the handbook.
But from another angle, what people today call handbooks, skills, operation manuals, experience docs, and SOPs are all pointing at the same thing: making human experience explicit.
This was not first noticed today. As early as 2023, people in our practice had already realized that if AI was truly going to enter the security workflow, eventually human experience, judgment paths, and analysis habits would have to move from implicit intuition into explicit text.
The problem was that most people at the time were neither good at this nor eager to do it. On the one hand, making experience explicit is genuinely hard. It requires people to take the judgments they have long relied on through intuition and habit and break them down step by step into something that can actually be explained. On the other hand, under the model capabilities and engineering paradigms of that time, the payoff from doing this work was not very direct. Many people instinctively resisted it because it felt tedious, inefficient, and like extra work with no immediate return.
But by 2026, an interesting reversal happened. Many people who were originally unwilling to make experience explicit started actively writing their own skills. On the surface, it looked like their habits had changed. But at a deeper level, the return structure had changed. Once coding agents became central, skills, handbooks, and SOPs were no longer just documents for people. They began to look more and more like natural-language programs that could directly shape an agent’s task decomposition, information use, execution stability, and output quality.
Put differently, the same act of making experience explicit looked like writing docs in 2023, but by 2026 it looked more like writing a program that directly takes effect. People did not suddenly become more diligent. They finally saw a direct and obvious payoff.
The foundation model companies will absorb this trend quickly as well. For example, Anthropic’s March 2026 update to skill-creator already started baking skill testing, benchmarks, regression checks, multi-agent parallel evaluation, comparator agents, and trigger timing optimization directly into the product:
https://claude.com/blog/improving-skill-creator-test-measure-and-refine-agent-skills
That perfectly supports the earlier judgment: once a capability is useful enough and general enough, foundation model companies will eventually turn it from a usage technique into a platform-native capability.
Why does this matter so much?
Because the real value of large models and agents is not that they create experience out of thin air. It is that they can efficiently execute, amplify, transfer, and reuse experience that has already been distilled.
When the way a senior security engineer works can be written down clearly, what to look at first, what to look at next, which features deserve suspicion, how certain attack patterns should be validated, and what the high-risk points are in particular business scenarios, then that body of experience becomes something an agent can automate.
From that perspective, the most important assets in AI security tooling may not be code alone. They may be knowledge accumulation, analysis paths, vulnerability pattern summaries, and the ability to express domain experience.
For security teams in particular, the vulnerability samples, attack patterns, and audit experience accumulated over the years are not just background material. They are the core fuel for building agent capability. In the agent era, the ability to express experience itself becomes a major competitive advantage.
At this point, a natural question appears: should we still use RAG, code slicing, and traditional tools? This is the part that most easily causes disagreement, which is exactly why it is worth stating clearly.
The answer is not “throw all of them away.” The answer is: they can still be used, but they should no longer sit at the center of system design.
Take RAG first. The essence of RAG is finding the needed content from a large body of material. That capability absolutely has value. The problem is that many people got into the habit of treating RAG as infrastructure that had to be built first, and then naturally poured large amounts of effort into deciding what to retrieve first, what to trim first, and what to assemble first.
In the Coding Agent era, the more reasonable idea is this: if you genuinely need to search across a large document corpus, then add retrieval. But if the agent can already get what it needs through grep, graph tools, scripts, or file search, then there is no reason to add a heavy RAG layer on top.
Even more importantly, the focus should not be on preassembling a standard-answer-style input for the agent. It should be on letting the agent directly access sufficiently complete and sufficiently raw material.
In other words, RAG is a usable capability. It is not a default premise, and it is not the center of development.
The same is true for code slicing. Code slicing and material slicing are common in academic work. The problem is not that they are useless in principle. The problem is that in real engineering they often run into several practical issues: slicing strategies are hard to design, they easily lose cross-file, cross-function, or cross-module information, the preprocessing cost is high, and as models improve, the marginal gain from this fine-grained preprocessing drops quickly. Many teams spend enormous amounts of time on slicing, only to realize in the end that the system did not become meaningfully stronger. It just became more complex, more fragile, and harder to maintain.
Traditional security tools still matter too. Capabilities like SAST, dynamic analysis, FAST, symbolic execution, and all kinds of validation tools still matter. But they should no longer dominate the entire main flow. They are better thought of as external tools the agent can call when necessary.
These capabilities can easily be exposed through MCP or similar mechanisms and then invoked by the agent when needed, in the way that best fits the situation.
That is a much more natural fit for the new generation of systems. The tools still matter, but their role has changed: they are no longer the skeleton of the whole pipeline. They are modular capabilities the agent can call on demand.
Once the methodology is clear, you eventually have to return to product choice itself. There is a very important product-level judgment here: code auditing is fundamentally a graph exploration problem, really an “unknown unknowns” problem.
What does that mean?
When a piece of code is handed to a model, it does not reason along a single path. It may enter from data flow, privilege boundaries, call chains, business logic, input validation, state consistency, economic models, and many other angles.
Some of those reasoning paths lead toward the right answer. Others lead toward false positives. Worse still, it is hard to know how many possible paths there are, whether some are still missing, and whether the search space has actually been exhausted.
That is why code auditing is inherently hard to complete.
A mature engineering mindset therefore should stop obsessing over whether everything has been found, and instead shift to a more practical target: convert an effectively infinite vulnerability reasoning space into a finite, reviewable, coverable task space.
That is a more grounded optimization goal: not theoretical completeness, but engineering coverage, replayability, and sustainable optimization.
By contrast, some other directions close the loop much more easily, for example large-scale log analysis, malicious IP mining, root cause analysis, and other security tasks with more deterministic answers.
Those tasks usually share one feature: the search space is smaller and the result is easier to validate.
That means the system can quickly form a loop of reasoning, comparison, reflection, and iteration.
A typical example is a task with clear ground truth. In that case, the agent can repeatedly produce an answer, compare it to the real result, reflect on where it was wrong, adjust the prompt or process, and try again.
Tasks like that are especially suitable for self-improvement, and it is much easier to get results from them quickly.
Go one step further and you see that not every agent system is suitable for automated iteration. Systems that are genuinely suitable for it usually share several features:
Their outputs can be compared, or at least there is some clear validation signal, constraint, or stage target.
Their intermediate process is observable, so the error does not only appear in the final result but can be localized to a particular decomposition, document, or sub-step.
Their unit of change is controllable, so you can modify prompts, skills, handbooks, strategy lines, or task templates locally instead of rebuilding the whole flow.
Their results are reasonably reproducible; otherwise every run becomes too random and automated iteration degrades into blind trial-and-error.
Put differently, automated iteration is not best suited to black-box, unobservable, unlocalizable, or non-modular agents. It is best suited to systems with clear phase boundaries, clear evaluation signals, sufficiently externalized state, and controlled change scope. Only in systems like that does feedback become meaningful, and only then can iteration accumulate into real capability.
From a product perspective, if a team is just starting out, prioritizing these easier-to-close-loop directions is often much more sensible than jumping straight into the hardest version of code auditing. In other words, it is usually better to make progress first in easier closed-loop scenarios than to challenge the hardest problem on day one.
This is another very practical question that often gets mixed together: how should AI security tools actually be evaluated, and are we evaluating a bug-hunting assistant or an auditing system?
A few days ago, when I was discussing this with Xiaohai Xu
, the third author of OpenAI’s EVMbench (
https://cdn.openai.com/evmbench/evmbench.pdf
), one point felt especially worth remembering.
From the client’s perspective, what they really care about is often not how many more medium or low findings a tool gets on average, but whether it can avoid missing the few issues that actually determine the outcome, especially high severity flaws that can directly lead to loss of funds. For many buyers, crit hit rate is the metric that feels closest to a purchase reason.
But that does not mean coverage is meaningless. For audit teams that already have strong human capability, AI tools look more like safety nets and amplifiers. For newer teams hoping to reduce cost and expand delivery capacity with AI, the tool is more easily expected to substitute or supplement human labor. Those are different settings, so the priority of the metrics naturally changes.
More importantly, crit is usually not something you can target directly. Many high-severity bugs are found only because the system has strong overall search ability, code understanding, path exploration, and coverage. In other words, medium and low coverage does not directly equal business value, but it often determines whether the system has any stable foundation for hitting crit at all.
So the more accurate framing is not coverage versus crit. It is this: coverage is the foundation, and crit is what decides the game.
Coverage determines long-term capability boundaries. Crit determines short-term market outcomes. A tool absolutely needs to keep raising its overall coverage. But what the market rewards in the short run is still the system that actually hits crit.
Push the question one step further, though, and another point appears: the reason coverage and crit are so often argued about may not just be about the metrics themselves. It may be because people are not evaluating the same thing.
If the goal is finding bugs, then what you are really evaluating is a high-efficiency executor.
From the perspective of vulnerability hunting, many of these judgments make perfect sense. In that setting, the human looks more like a director: deciding the direction, cutting the attack surface, defining the threat model, and deciding which slice is worth digging into. The AI looks more like a high-efficiency executor, rapidly exploring along that direction, constructing PoCs, validating exploitability, and trying to surface the most valuable vulnerabilities.
If you place ToB security researcher Devansh’s article, “Needle in the haystack: LLMs for vulnerability research” (
https://devansh.bearblog.dev/needle-in-the-haystack/
), inside this frame, it fits this bug-hunting perspective quite closely. The point of that article is not how to produce a more complete security judgment on a project. It is how a researcher gives direction, a threat model, and a slice first, then lets AI act as a high-efficiency executor for focused exploration, PoC construction, and high-value vulnerability hunting.
In that frame, the most important thing is not whether the system steadily covers the whole project. It is whether the system can hit as many valuable bugs as possible in limited time, especially crit. If it can keep surfacing the few issues that matter most, it is already highly competitive.
From that perspective, a methodology that emphasizes light scaffolding, focused exploration, local deep dives, and PoC-driven reasoning is not wrong at all. It is simply optimizing for how to hit high-value bugs faster, not how to produce a more complete security judgment.
But if the goal is auditing, then what you are really evaluating is a team that can hold the floor.
From an auditing perspective, the question is no longer just “can this system dig out a beautiful high-severity bug this time?” It becomes: can this system keep accumulating capability, hold the floor, avoid missing key risks as much as possible, and produce a more stable security judgment?
Here, the human looks more like a manager, not just a director. What matters is not only pointing AI toward one high-signal direction and letting it sprint. What matters is how tasks are divided, how coverage is raised, how results across modules are combined, how one discovery becomes future capability, and how the system remains replayable, regressable, and explainable across repeated runs.
Put differently, what you need here is no longer just a high-efficiency executor. You need something more like a team that can push the work forward in a stable way. It needs planning, coverage, handbooks, checklists, verifiers, regression, and ledgers. It must be able to turn local discoveries into system-level capability, instead of relying every time on a single flash of insight inside one high-signal slice.
And that leads to another very important piece of human experience that is easy to overlook: putting a group of efficient executors together does not automatically give you a good team.
A strong vulnerability researcher working with a strong AI executor can absolutely produce beautiful results quickly. But putting many such efficient executors together does not automatically give you a good audit team. A team still needs division of labor, scheduling, handoff, floor guarantees, postmortem, regression, knowledge sharing, and quality control.
The same is true for AI systems.
If a system only shines on a handful of high-signal points but cannot steadily cover the attack surface, cannot preserve intermediate state, cannot explain why it found something this time and not last time, and cannot turn discoveries into the next round of planning and verification, then it looks more like a strong executor than a truly dependable auditing system.
So the relationship between coverage and crit often has to be understood at one layer above the metrics themselves:
Are we evaluating a bug-hunting assistant, or an auditing system that can preserve the floor and gradually form team-level capability?
If the goal is more bug hunting, then crit hit rate is naturally closer to value. If the goal is more auditing, then coverage, floor quality, replayability, regression ability, and cumulative capability become much more important. A mature system, of course, ultimately needs both. But the default optimization direction for those two goals is not exactly the same.
If you take the above judgment and apply it to different tracks, a very real difference appears: Web2 and Web3 code auditing are not problems of the same scale.
In traditional security settings, Go, Java, C/C++, operating system components, large open-source projects, models often have an easier time producing useful results. The reasons are straightforward: the vulnerability patterns are more mature, the code corpora are larger, audit experience is more standardized, and the toolchains are more complete.
So in Web2 settings, models do tend to produce useful results more easily.
But once you move into Web3, contract security, and complex financial-logic vulnerabilities, the problem gets much harder very quickly.
Because these vulnerabilities are often not simple memory issues or input validation bugs. They are specific business process flaws, defects in complex fund-flow logic, exploit paths embedded in financial operations, or issues in the economic model itself.
They are highly dependent on domain knowledge and highly dependent on accumulated experience.
That means they are harder to standardize, harder to automate, and harder to build around, but also harder to copy.
From a competitive perspective, that is actually good news. The harder the direction, the less likely it is to be easily replicated. That is also why two things that are both called code auditing can have completely different methodologies, engineering complexity, and output rhythms depending on the track.
If you bring the methodology above down into engineering implementation, I do not want to expand too much here. Part of it touches core methodology, and part of it simply should not be described too explicitly in a public article. The only thing worth saying is this: a truly effective system is not one with a locally optimal point. It is one that, as a whole, can both preserve the floor and dig deeply into the tasks that matter.
In my view, a genuinely valuable agent needs at least two things. First, an architecture that is light, loosely coupled, and replaceable enough to evolve naturally as models improve. Second, an execution process that is stable, reproducible, and preserves as many intermediate results as possible. Only then can the system know why it succeeded this time, why it failed last time, and how to keep iterating.
A very intuitive example is the difference between
and
(
) for attack-transaction RCA. At similar levels of accuracy, the former is usually much faster, while the Clara route is much slower and can even require six hours of setup.
But this is not to say Clara is bad. It is to show that different routes make different trade-offs. Clara absorbs a large amount of bad-case correction into the runtime itself, so humans do not have to intervene case by case. The other route is faster, but depends much more on humans anticipating bad cases in advance and continually writing them back into documents and templates.
If you want to understand
in more detail, you can read Professor
‘s article on that direction.
From a more engineering angle, automated iteration is very much like an electrical closed loop. Logs, intermediate results, validators, and human review results are the sensors. Target outputs, ground truth, and rule constraints are the reference input. The gap between them determines what the system should change in the next round.
The key idea in that analogy is simple: do not rewrite the whole system every time a problem appears. Keep the main structure stable, externalize intermediate state, localize errors to concrete levels, and limit the scope of each iteration. That is how feedback becomes meaningful, and how iteration starts turning into capability.
Push the idea one step further, and you get another judgment: an agent system that needs to iterate quickly should, as much as possible, be designed around a single bottleneck.
What single bottleneck means here is not that the system literally has only one limiting factor. It means that most problems should converge to a small number of clearly identifiable key links, instead of being scattered across every corner of the execution chain.
Its value is not only that it can be optimized more easily. It is that problems become easier to locate, reproduce, and fix. A lot of the real difference in outcomes is not produced at the last execution step. It is determined much earlier.
I had already realized this when I was building Finite Monkey Engine:
https://github.com/BradMoonUESTC/finite-monkey-engine
At the time, my understanding leaned more toward making open-ended exploration and strong convergence themselves into the bottleneck. Looking back, that direction was not exactly wrong. But what matters more to me now is this: the bottleneck should not only be strong; it should also be observable, replayable, and iteratable.
For audit systems that need to improve continuously, this matters a lot. The system does not just need to be strong. It also needs to be able to close itself up.
This is one of the most important implications of the paradigm shift.
If more and more people can rely on Coding Agents to complete development, then the real gap between teams will no longer be just who writes code faster. It will be who understands business problems better, who decomposes tasks better, who distills experience better, who designs more iterable execution paths, and who is better at turning complex problems into natural-language programs an agent can execute.
From that angle, competition in AI security tooling may increasingly look like this:
Can you distill the way an expert works into a method that is reusable, automatable, and continuously evolvable?
That is also why companies or individuals who only pivoted fully toward LLMs in 2026 and expect to solve everything through one big organizational sprint are unlikely to build anything with long-term value. Without enough accumulation, without a deep understanding of the tendencies and probabilistic nature of LLMs, and without understanding product boundaries, user needs, and deployment constraints, what they build is usually just demos, reports, and slides, not products that can run long term and create real value.
To put it even more directly, the real competition in the agent era is not who catches up faster. It is who understands more deeply, accumulated earlier, and sedimented more thoroughly.
If the whole article is compressed into a few practical principles, it boils down to something like this.
Do not think about frameworks first. Think about needs and information accessibility first. Figure out what problem you are actually trying to solve, and only then decide whether you need RAG, SAST, dynamic analysis, code slicing, or workflow frameworks. Rather than designing a complex information assembly flow at the outset, first ask: what material does the agent need to see, and how can that material be exposed as completely as possible?
Get humans and agents working together first, then automate. Do not rush to write the whole system. Let humans and tools like Cursor, Codex, and Claude Code work through the task together first. Once the flow works smoothly, then automate it.
Write the experience down. What can really be accumulated into capability is not just code, but handbooks, skills, SOPs, vulnerability pattern summaries, and business-experience documents. The clearer you can express experience, the easier it becomes to turn that ability into agent capability.
Prioritize tasks that close the loop easily. If a team is just starting out, prioritize directions with smaller search spaces and easier validation. Make something work first, then gradually move toward harder problems like code auditing.
Design for the next six months, not for the last year. Model capability is still changing rapidly. Do not build a system for yesterday’s constraints that will already be obsolete tomorrow. A future-facing agent needs at least two properties: first, it should be light and flexible enough to evolve naturally as models improve instead of requiring complete rewrites; second, its workflow should be stable, reproducible, and preserve clear intermediate results, so later feedback, iteration, and targeted optimization have something to grab onto.
There is also another important reality: foundation model companies themselves are iterating aggressively. Anything that proves valuable enough and general enough will eventually be built into platforms, toolchains, or even model defaults. Rather than chasing the toolchain of the moment, it is better to chase the underlying principles. Skills, Memory, and Subagents are all examples of this: first they proved themselves in practice, then they were productized, platformized, and built in.
Instead of making systems heavier and heavier, it is usually better to make them lighter, more flexible, and more able to evolve alongside agent capability.
If I had to summarize the core point of this article in one sentence, it would be this:
When building AI security tools today, the most important thing is no longer how to assemble the framework, but how to distill the human analysis process and let Coding Agents reproduce, execute, and iterate on it with sufficiently rich material.
This is not a minor improvement. It is a change in R&D paradigm.
The old way, first build the framework, then wire up the tools, then connect the model, was not completely wrong. It is just increasingly a previous-generation method.
The newer method starts from needs, from the human process, from experience sedimentation, and from information accessibility, while carrying stronger agent capability on a lighter code skeleton.
For the security industry, this shift deserves special attention. Security has never been just about piling up technology. It depends deeply on experience, paths, judgment, and postmortem ability. Whoever turns those capabilities into forms that agents can execute earlier has a better chance of gaining the initiative in the next wave of tooling evolution.
Ultimately, the real dividing line may not be whether you know how to build AI security tools at all. It may be this:
Did you build another old-era workflow system, or a security system that truly belongs to the agent era?
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.