Build, Buy, or Borrow: How Founders are Thinking about their LLM Stacks

This is our second installment of Fields Notes, a new series that takes you behind the scenes of the conversations we’re having with founders and operators in the Adverb community. Each article will bring a practical, candid perspective from the people building, scaling, and learning in real time.

When Sara Beykpour started building Particle News, an AI-native news platform that uses LLMs to summarize and personalize news coverage, in early 2023, there was no playbook—no dashboards, no prompt libraries, no evaluation tools to measure LLM output quality. Particle had to build nearly everything from scratch just to get their product out the door.

Two years later, the AI tool options look entirely different. Startups can now spin up evals, caching, and fine-tuning with off-the-shelf tools. But as Sara and Jennifer Jang, founder of StoryGold—which uses generative models to design CAD for jewelry—discussed at our recent Adverb Founder Summit, there’s still one fundamental question every founder building with AI must answer: What do you build yourself, and what do you buy?

The internal tools that Sara and the founding Particle team built early on became deeply intertwined with their product architecture. “It’s so tied into our data model and caching systems now that it’s hard for a third party to beat that,” Sara explained.

That deep integration has become a competitive advantage—the control and autonomy compounded over time. Jennifer, however, learned the build vs. buy tradeoff firsthand at StoryGold. They began with off-the-shelf tools, then moved away from them after realizing that they wanted to have more control around how our eval pipeline integrated with the product.

Both Sara and Jennifer reflected how the landscape has shifted. “OpenAI’s latest suite connects evals, tracing, and fine-tuning in a way none of the early tools did,” Sara said. “If you’re starting now, the baseline is so much higher—you’d have to justify building what OpenAI will do better in three months.”

The new tradeoff isn’t just build vs. buy—it’s control vs. velocity. Mature teams with entrenched infrastructure are slower but have more autonomy over their stack because they’re built from scratch, while early-stage startups can stand on the shoulders of integrated ecosystems and sprint. To help navigate this decision, Jennifer offered the advice that: “If inference or evaluation is critical to your core experience, you want to own it early. That’s where your differentiation lives. You don’t want to outsource your edge.”

Particle’s internal evaluation (“eval”) system, which determine the quality of output from an LLM, evolved through trial and error into a few practical rules every AI product builder—should know:

1. Tell the model how it will be graded. Telling the model how its output will be evaluated measurably improves performance. LLMs try to predict what a “good” answer looks like. If you’re clear—concise, factual, friendly, accurate—the model steers toward that target.

2. Order matters—ask for the thinking before the answer. LLMs generate text one token at a time. If you ask for the conclusion first, they’ll often “lock in” and then justify it. Flip that order and ask for the reasoning before the decision. “We got much more consistent answers once we changed our prompts to reason first,” Sara explained. “It sounds small, but it changes how the model thinks.” For example:

• Answer-first (bad): “Is this comment bullying? Answer Yes or No. Then explain why.” Result you might get: the model says “Yes” and then invents shaky reasons to match that yes.

• Reasoning-first (good): “Explain why this comment might or might not be bullying. After your explanation, conclude: Yes or No.” Result you’re more likely to get: a thoughtful explanation that actually supports the final decision.

3. Match structure to logic. Output fields should follow a logical flow—evidence first, summary second, verdict last. “Think of it like a reasoning chain,” Sara said. “Each step feeds the next. That makes it much easier to debug when something goes off the rails.”

Sara and Jennifer posited a key distinction between testing: evals prevent regressions; A/B tests drive innovation.

Evals act as guardrails, catching prompt breakage and measuring stability. “If your accuracy drops from 80% to 50%, you see it immediately,” Jennifer said. “Evals are like automated tests for your AI—they tell you if you’ve broken something you already knew worked.”

A/B tests, by contrast, measure real-world impact. “They show you what actually resonates with users,” she explained. “We’d run two prompts in production and look at downstream metrics—engagement, speed, satisfaction. You learn things evals can’t capture.”

As Sara summarized: “Evals keep your baseline solid. A/Bs help you climb higher.”

Live comparisons to surface meaningful behavioral differences and customer impact.

As context windows balloon, retrieval-augmented generation (RAG) become essential for accuracy and cost control. If you’re passing in hundreds of thousands of tokens—the basic building blocks of how language models (like GPT) read and generate text—the model gets diluted and confused. But, a good RAG system improves focus and consistency.

“RAG improves accuracy because if you’re passing in hundreds of thousands of tokens, the model is focused equally on everything that you’re passing in. And the model doesn’t do a good job with finding the right context within the context,” Jennifer said. RAG works by feeding the model only the most relevant information—reducing noise, token bloat, and hallucinations.

Two years in, the LLM toolchain is finally catching up to early adopters. Founders today can choose between tightly integrated commercial stacks and bespoke systems with deep in-house control.

The right path depends less on philosophy and more on context—your data, your scale, and your team’s technical leverage.

As Jennifer put it, “We used to build everything because nothing existed. Now the question is: what’s still worth owning?”