There has been lots of chatting about software supply chain security recently, motivated by popular package exploits.</p>
Well, I have some relevant news: cargo-crev</a> now supports LLM-assisted code reviews. Go try it!</p>
Read on to get more information and background.</p>
I started working on Crev in 2018. The idea
was (is) simple: if each of us (developers) reviewed at least some of our
dependencies, and then we could share and distribute such reviews
between each other with the help of a Web of Trust, we
could get a good review coverage of the ecosystem, gain more
trust in our supply chains, report back issues upstream and
improve overall health of the open source ecosystem.</p>
If I may say so, on a technical level I'm satisfied
with the UX and flow that was achieved. However, around 2020
I lost my enthusiasm for this project.</p>
Why? Because it became apparent that no matter how well
Reviewing code, even superficially, is a lot of effort and
it takes a very long time. And it
does not feel nearly as satisfying as actually creating
something new — writing code.</p>
Open Source community was already overburdened with
just maintaining code. Asking developers to take on
yet another unpaid responsibility of trying to secure
supply chains is just asking too much.</p>
Just a few weeks ago I was reading some articles about new LLM
models finding non-trivial security issues, and Linux kernel and curl
developers admitting that after a deluge of mostly worthless slop security reports
they used to complain about, now they tend to receive actually worthwhile
AI-assisted bug and security reports. It reminded me about I'm not trying to overhype LLMs. But the fact is that
they can do, and in high volume, what developers themselves have
no time for: the 90/10 security scanning that was otherwise quite hard
to automate.</p>
An LLM can easily and reliably check if a code version published
on https://crates.io matches the code published in git.</p>
An LLM can easily scan It is actually very hard to hide key-stealing malware
in a package that was supposed to format units, etc.</p>
Especially in Rust, doing things that are wrong or
out of place creates a lot of noise, making such code
easy to notice, even by an LLM reviewer.</p>
It might not be a silver bullet, but it is definitely
better than doing nothing.</p>
Note: In the initial release Since version 0.27 which will start the agent 10 times, each time
selecting and reviewing a single dependency.</p>
The agent will produce&update a single shell script
that can be used to conveniently review and sign
all reviews.</p>
While the above is meant as a standard mass-review flow,
the core built-in agent review skill is available as an output of:</p>
and it should be easy for anyone to modify it and/or build their own
LLM-assisted workflows.</p>
The core change is that Crev's reviews now have
fields to indicate that an LLM was used for the review</a>.</p>
The rest is just relatively minor functionality to make producing
LLM reviews convenient end to end.</p>
For people skeptical of LLMs, options to ignore LLM-generated reviews
have been and will be added where appropriate. You can just
ignore the slop reviews if you don't trust them, fine with me.</p>
While working on this feature and testing it myself, I have produced
quite a few LLM-assisted reviews</a>.
Judge by yourself.</p>
To me these meet the bar of being useful. And they turned some spare
capacity from my Claude subscription into something that I otherwise
would not be able to do myself.</p>
This is only an initial attempt at harnessing the AI in cargo-crev.
There still might be lots of things to improve and extend, but we have
to start somewhere.</p>
If you like the idea and find it promising, I encourage you
to try it out, give some feedback, and submit improvements.</p>History#</a>
</h2>
cargo-crev</code> was an attempt to implement such a system,
for the language ecosystem I cared about most — Rust.</p>
cargo-crev</code> actually works, the biggest obstacle to fully
realize the idea was lack of time by developers.</p>
LLMs getting good at finding issues#</a>
</h2>
cargo-crev</code>
and I realized that AI can actually fill the gap that made me
doubt it.</p>
build.rs</code> and the rest of the code and
look if anything looks out of place.</p>
How to use it#</a>
</h2>
cargo-crev</code> supports
only Claude Code agent. If you're interested in adding
support for other coding agents, it should be relatively
easy — most scaffolding is already there. Feel free to chat
and create a PR.</p>
cargo-crev</code> has a built-in
review loop.</p>
cargo crev ai review-loop --iterations 10</span></span></code></pre>
cargo crev ai skill review</span></span></code></pre>
How it works#</a>
</h2>
How well</em> it works#</a>
</h2>
Summary#</a>
</h2>
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.