2025 LLM Year in Review

Welcome to 2026! Nothing better to kickstart the new year than to look back at where we left 2025.

While talks about the impending burst of the AI bubble continue to mount, last year was still dominated by continuous news on models, tools, and ways to integrate AI even further.

Andrej Karpathy provided his take on the highlights in the LLM world:

• Reinforcement Learning from Verifiable Rewards

Up until 2024, the recipe for LLM involved pretraining, supervised finetuning, and reinforcement learning from human feedback. As of 2025, RLVR has been used more and more.

The result is that the strategies the LLM develops resemble human reasoning. More and more models rely on this approach and offer a lever to control how much “thinking” is involved when providing the answers.

• Assessing Models Becomes Harder

Keeping true to the nature of benchmarks, assessing how good a model is not as easy as before. It is not uncommon to see results being challenged and training your model to ace the benchmarks as a goal!

• The rise of LLM apps

Cursor, and other similar tools, more and more bundle and orchestrate LLM calls on your behalf: from context engineering, using multiple models to balance cost and performance, to offering autonomy controls (how much you trust to do things on its own)

• Vibe Coding

LLMs and surrounding tooling have evolved enough to enable you to create real applications almost exclusively from English. While it is still dangerous if taken directly to production without proper vetting, it is a big enabler of many uses where its shortcomings are acceptable.

My take remains the same; it is a valuable tool that should be used as long as you remain in control of the quality of what goes into production.

As expected, the vibe coding term lost its initial meaning and is used loosely to indicate the workflow where you specify what you want in natural language, and the LLM will generate most, if not all, code for you.

With that definition, the “good” vibe coding is when you at least review the generated code and address security, standards, and performance issues for real production applications. In this world, we start seeing productivity being measured by how fast you can generate the application.

This article proposes that we should consider introducing Big-O as part of our metrics. The idea is that vibe complexity is real and can affect productivity, as two different approaches can consume different numbers of tokens and require (or not) repeated retries to get where you need to be.

For example, I covered Specification Driven Development in this article. While valuable, I noticed that it consumes a huge amount of tokens due to the many stages and artifacts that are generated and constantly passed as part of the context. In contrast, if you choose the Plan mode in Visual Studio Code/Copilot, you can reach similar quality levels for smaller features.

With that in mind, we should expect companies to start pivoting from a raw code generation focus to one where the engineering value is also measured by the consumed tokens needed to achieve the goals.

Remember that more concrete and specific prompts lead to fewer retries vs generic instructions.

New Year’s resolutions are usually broken by the first week, but one that I keep is to look back at the previous year from many aspects and reflect on things that I would like to explore, change, or abandon.

This article presents some reflections that resonate with mine, including:

• Programmers’ resistance to AI has lowered

It is increasingly rare to find developers who do not use some form of AI to help with research or code generation.

Now the next step is to adjust your workflow to explore novel ways to bring more value to you.

• Transformers technology will not lead to AGI

All that I read points to diminishing returns of the transformer technology. Every new model is usually better than its predecessor by a few percentage points.

I see most perceivable gains coming from tooling that orchestrates those calls and development workflows that use what is available in novel or easier ways.

• Chain of Thought is “standard”

More and more models use CoT as a way to improve the output quality. The idea is that the intermediate reasoning steps break complex problems, described in the prompt, enabling error detection and self-correction.

Graphite was bought by Cursor, but if you are interested in the story behind how it got there, I would recommend checking this article. It is a semi-external testimony about their trajectory and some comments on how they choose to operate.

This is a reminder that the path to success, whatever you define as such, is hardly ever a straight line, and anyone wanting to venture into creating something new needs to be ready for the ups and downs.

It is important to persevere but not be afraid to pivot and fine-tune your approach/product. For example, they decided to favor in-person collaboration over remote-only, or how they split the responsibilities among the three founders: CTO, Experiments, and Growth.