🔗 Learn more about me, my work and how to stay in touch: maeste.it: personal bio, projects and social links.
A week where the links section is full of new frontier models, and I found myself making a choice. I could have done the deep dive on one of those, and I was tempted, but I chose to focus on the importance of the harness, the software scaffolding around the model. Not because models don’t matter, quite the opposite. The two things don’t exclude each other, and both are decisive for the agentic future. But there’s a difference that strikes me as fundamental. On models, we’re living the race to the most incredible model of them all, and that’s something we expect, however spectacular it is: every week there’s a new SOTA, and we’ve learned to barely be surprised anymore. On the software engineering behind agents, instead, we’re at the ChatGPT moment: something is changing so fast, and so much in plain sight, that in a year it’ll feel obvious and today we still struggle to name it.
In the deep dive I prove it with numbers, because this time the numbers are there. Two charts say the same thing: at parity of model, changing the harness moves the result more than we’d expect. And Lilian Weng, with a post I devoured, gives us the definition and the direction of travel. My thesis stays the same, but now it has the backing of research and data: whether you use a closed model or an open weight one, the real lever is the harness, and that’s where I’m betting everything, with Lince.
In the links section you’ll find exactly the frontier models I was talking about: OpenAI’s new GPT-5.6 family and GPT-Live, Meta’s Muse Spark 1.1 and Muse Image, Grok 4.5, Tencent’s Hy3, ByteDance’s Seedream, and Google’s Gemma 4. Enjoy the read.
My agenda
Saturday saw the new episode drop: Fable is on another level, and people are using it to optimize the harness and teach Opus.
Plus the real costs of models, the AI China-US cold war, and LongCat on Huawei chips. Listen
Our projects Lince.sh and AntiVocale (Google Play, GitHub), you know them well by now. Also take a look at Agent ready skills, which I talked about on June 24 at AIConf.
We’re thinking of doing short livestreams on YouTube, X and maybe LinkedIn, once a week, around lunchtime, to show you practical things about our projects, personal agents, and different uses of AI. First experiment this Thursday, follow us on social to find out more
On my own:
Finally a quiet stretch for my public appearances. After all, summer is here, but we’re already working on September.
As soon as the videos from the conferences of the past few months come out, I’ll flag them, because I’d love your feedback.
The harness matters more than the model: now there are numbers to prove it
I’ll start with two figures that kept me glued to the screen. The first: same task, same model, and by changing only the harness, the cost per task drops from $0.21 to $0.12, the time from 48 to 27 seconds, the tokens from 14,200 to 8,800. Down 41%, down 44%, down 38%, without touching a single weight.
The second is even nastier. In the cost-versus-pass-rate chart, the same Opus 4.8 jumps from 87 to 90 percent just because what changes is the harness and the effort level, at equal spend.
These are two images that say the same thing, and it’s exactly my pet obsession. Anyone who’s been following me for a few weeks knows it: from the newsletter where I argued that the agent is a process running in a second-level operating system, to the one where I quoted Simone Basso, “the model is a commodity, the harness is the asset.” Now putting it down in black and white, with a definition to boot, is Lilian Weng, in a post I really recommend. For her, the harness is the system around the model that orchestrates execution: it decides how it thinks and plans, how it calls tools and acts, how it perceives and manages context, how it stores artifacts and evaluates results. And her short-term prediction is blunt: recursive self-improvement doesn’t start from a model rewriting its own weights, but from harness engineering itself. The analogy she uses is the one with the operating system, the same one I’d arrived at. It’s gratifying.
The point that interests me most is brought into focus by two articles that came out almost in parallel on X, and they converge on a three-level classification: model, harness, artifact. Shilong Liu puts it this way: evolution can happen inside the model, inside the harness, or in the artifacts the agent produces. pirroh of Replit arrives at the same place calling the third level “context,” but the substance doesn’t change. And here comes the part that hits close to home for me. pirroh is explicit: at the frontier, with models like Fable 5 or GPT 5.6, you don’t own the weights and you can’t fine-tune them. What you actually control is the harness, meaning improving code, tools, and instructions by mining production traces, and the context, personalizing per agent, user, and organization.
It’s the mirror image of the thesis I defend, as a European citizen, on open weights. Whether you use a closed model or an open one, the real lever remains the harness: except that in the closed case it’s the only one you have left, in the open one you have all of it. And it’s a lever that compounds, you ship it every day and it improves with every interaction.
Careful, though, there’s a caveat I don’t want to hide. Weng reports an old result from STOP, a recursive self-improvement experiment that with GPT-4 improved performance but with weaker models like GPT-3.5 and Mixtral degraded it. The recursive structure alone isn’t enough: the base model has to be capable enough to improve the mechanism. Improving the harness lets you get more out of the same model, but intelligence remains the core. And then there’s the most important design rule of all, the one I’m working on with Lince: the evaluator and permission control must sit outside the loop that evolves the harness. If you let the program modify itself inside the ring, you break the abstraction boundaries and open the door to reward hacking. It’s governance, again.
And here I come back to the thing I keep repeating. All these self-improvement loops give their best where results are verifiable. Weng says it, loops work better when metrics are measurable and objective; Shilong sums it up with three perfect questions, what evolves, what feedback drives it, where the loop closes; and pirroh practices it with ViBench, his benchmark for vibe coding. If the loop closes on code or on an eval, you have something solid to hold onto; if it closes on marketing, you’re optimizing toward nothing.
I’ll close with Matei Zaharia, who measured coding agents on Databricks’s real internal tasks: there are surprising opportunities to lower costs and raise quality, and open source models are now truly competitive. It’s no coincidence. The lever isn’t waiting for the next flagship model. It’s the harness, and that’s where I’m betting everything.
Links that caught my attention this week
Meta: agentic reasoning and images
Spark 1.1 is Meta Superintelligence Labs’ multimodal reasoning model, built for agentic tasks, with actively managed context up to one million tokens and multi-agent orchestration. Muse Image is the lab’s first image generation model, with readable text in the image and direct editing from sketches.
Meta is getting serious about agentic. What interests me about Spark is that the active one-million-token context and multi-agent orchestration are native: it speaks the harness’s language, not the chatbot’s. And Muse Image, with readable text, hits a point where old generators always got it wrong.
OpenAI: the new frontier and the voice
GPT-5.6 in three tiers, Sol, Terra, and Luna. Sol is the new flagship, SOTA on coding, cybersecurity, and science, and on Agents’ Last Exam it beats Claude Fable 5 by over 13 points at about a quarter of the cost, with parallel agents in “ultra” mode. GPT-Live is the new generation of full-duplex voice models that listen and speak at the same time.
Sol is the new benchmark, but the number that matters isn’t the +13 over Fable, it’s the cost at a quarter: the frontier is increasingly bought in installments. What intrigues me about GPT-Live is the pattern, delegating to reasoning in the background while the voice stays alive: it’s orchestration, again.
Tencent’s Hy3
An open source Apache 2.0 MoE with 295 billion parameters, 21 active, 256K context, that competes with open flagships that have two to five times its parameters.
This reinforces my European thesis head-on. An open Chinese MoE that holds its own against much larger models means the alternative to the frontier exists, and it’s the insurance policy I’ve been talking about. Free to try on OpenRouter until July 21.
Grok 4.5
xAI’s model trained with Cursor, 80 tokens per second, roughly double token efficiency, default in Grok Build, in Cursor and via API. Not yet in the EU, expected mid-July.
Two notes. The 80 tokens per second is music for multi-agent systems, where latency between calls adds up. And the EU left out, again: every model that arrives late for us is another push toward open weights.
ByteDance’s Seedream 5.0 Pro
A multimodal image creation model built for production design: complex layouts, precision per-region editing, layer separation, native text in 14 languages.
What strikes me isn’t the image quality, it’s the target. Per-region editing and layers are pipeline features, not chatbot ones: when components become composable, it makes more sense to assemble them into a multi-model system.
Gemma 4
A new generation of open weight multimodal LLMs, dense and MoE from 2.3 to 31 billion, a “thinking” mode, long-context efficiency, and a unified 12B model that processes raw audio and images. Apache 2.0.
Google keeps pushing on open and on local, and the unified 12B that ingests raw audio and images without an encoder is the most interesting piece. Open weights, Apache 2.0: more ammunition for the insurance policy.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.