Purusottam Mupunu

Cloudanix • 6K followers

LLMs are amazing at generating code. However, one of the biggest limitation of this code isn’t intelligence - it’s lack of context. Things get interesting (and messy) when it comes to large codebases. It falls apart and it's painfully obvious why. LLMs can generate working implementations from a high-quality prompt. At their best, they can produce a correct feature in one shot (“one-shotting”). This helps dramatically speed up development for clearly scoped tasks. This helps with: ✅ Productivity boost for implementation-heavy tasks - For repetitive CRUD operations or isolated scripts, LLMs can collapse hours of work to minutes. ✅ Enhanced prototyping — They help quickly explore solutions, enabling engineers to iterate design ideas faster. ✅ Lower entry barrier — Less context switching (e.g., from editor to docs), faster feedback loops. But these benefits fade when tasks require deep architectural knowledge, modular context, or long dependency chains, especially in large codebases. In this blog post Kieran Gill from Blueberry Pediatrics highlights some of the reasons it fails and how it can be improved. Here are a few reasons why LLMs fail with large codebases: ❌ Lack of global context: LLMs don’t “understand” the whole system - they only see what’s in the prompt. ❌ Hallucinations in unfamiliar parts: When missing architectural knowledge, LLMs may generate incorrect code, leading to bugs or regressions. ❌ Rework becomes more expensive than doing it manually: Failing to one-shot means repeated rounds — which erodes the time savings. Here's how you can solve the challenges: ▶️ Guide LLM using Prompt libraries that show Architecture, best practices, domain knowledge, etc. ▶️ Improve the codebase by making code modular, consistency in naming, and keep code clean. ▶️ Invest in review & automation for verification of LLM's design choices and end product (code, unit tests, etc.) In large codebases, the real productivity is gained by guidance + oversight + modular architecture. #ai #llm #aiengineering #engineering https://lnkd.in/gFzFunEV

Mohit Sharma

Retensa Employee Retention • 1K followers

Do LLMs actually update their beliefs, or just guess? (And why it matters for your business) I came across an interesting study recently that got me thinking about how LLMs handle uncertainty, and honestly, the findings were a bit surprising. The researchers tested a range of LLMs on a simple recommendation task, predicting which flight a user would pick based on their past choices. You'd expect the model to get better with each round as it learns more about the user. But most LLMs plateaued after just one interaction. They weren't really learning from new information they were essentially making the same guess over and over. This is where Bayesian reasoning comes in. The idea is simple: you start with a belief (prior), you see new evidence, and you update(this becomes your new prior). Repeat. It sounds obvious, but most LLMs don't naturally do this. They lack the ability to maintain and refine probabilistic estimates across multiple interactions. So the researchers tried something clever, instead of training the LLMs on correct answers, they trained them to mimic the decisions of an optimal Bayesian model. The difference matters. Learning from perfect answers teaches the model what to think. Learning from a Bayesian model teaches it how to think under uncertainty. The results – not only did performance improve significantly on the original task, but the models transferred that reasoning ability to entirely new domains they'd never seen like web shopping recommendations. ⚡ What does this mean for your business? If your use case requires a model that can transfer knowledge across domains like adapting from customer service interactions to product recommendations this research suggests Bayesian teaching could meaningfully improve your LLM's performance. Rather than static pattern-matchers, these models become adaptive agents capable of cross-domain reasoning, internalizing the core principles of Bayesian inference without needing to be retrained from scratch on every new domain. You save cost, time, and can pick up a new hobby. Still early research, but a promising direction. Link: https://lnkd.in/eGJYRqSG

Robyn Le Sueur

ADVANTIQ • 3K followers

I realise I talk a lot about what models I like for powering my agentic IDE, but don't talk a lot about the models I actually build with, so let's go into that a little For actual production use cases (at least mine), particularly agentic ones, you want efficiency. These are going to be workhorses, not chatbots, so frontier models costing huge sums for a million tokens are simply not worth it. Large context is often useful, especially when handling large documents, but we can work around this with techniques like rolling summaries if needed. Speed is also an important consideration, we need to handle millions of requests a week, so the model can't spend 10 minutes thinking about each one like it's GPT-5.2. That doesn't mean reasoning models are completely out, but make sure you're using one that knows when to stop thinking for simple requests, or you can specify low reasoning. So what models do I favour having said all this? Believe it or not gpt-4o-mini, gpt-4.1-mini and gpt-4.1-nano, these are my unsung hero's that despite getting next to no attention, provide that balance of speed, efficiency, accuracy and in the case of 4.1, context length that's perfect for high load production use cases. I would honestly recommend when building anything with AI, start with a tiny model like 4.1-nano, see if it can handle it, and only move up to something bigger if testing shows it can't handle the use case. Too many projects start with the biggest frontier model they can find and never consider they could do it cheaper and faster with a smaller model, leaving people with the idea that AI is prohibitively expensive. It's not, you just need the right tool for the right task, not the biggest tool you can find.

Charles Higgins

Redberry Labs • 2K followers

A couple of years ago, I gave a talk where I suggested the focus on LLMs could be Nvidia’s undoing. I was obviously spectacularly, ridiculously wrong... but part of me wonders if the argument has some merit, but the timing was off… My reasoning: LLMs aren't AI's final form. Token-based autoregression has clear limits. Nvidia was over-optimizing hardware for LLMs. Sigmoid curves look exponential at the start—we're likely in the diminishing returns phase of scaling. Fast forward to now (or last month.. anyway): Yann LeCun yet again calls auto-regressive LLMs "doomed." Each token compounds error probability exponentially. "Not fixable without a major redesign." He's pushing JEPA as an alternative (link in the comments for the curious — cool read). So what? Well, Nvidia (and the industry as a whole) have spent billions optimizing GPUs for transformers. If transformers hit a ceiling and we need a major architectural shift (like CNNs → transformers), current GPUs will devalue faster than depreciation schedules suggest. Nvidia and by extension much of the AI compute ecosystem could take a serious hit. For those of us concerned with AI risks (both practical and financial), this cuts both ways: Upside: There's massive value in building safety systems around current, and future generations of AI models. AI will always have problems, but new architectures will bring new solutions—and entirely new problems. De-risking AI at all levels of the stack will likely continue to be difference between PoC and production. The ride has just started.  Downside: If Nvidia crashes, AI funding tightens up. Unlikely to be an extinction level event, but will likely thin the herd of startups, and potentially concentrate the AI winners to a few large platforms. Either way — I stood up and proposed selling Nvidia in late 2023. Please form an orderly queue for more financial advice. https://lnkd.in/e2cGHPFj