Welcome to the 62nd issue of AI Agents Simplified 🍻
This issue is brought to you by Novu Connect
Welcome to the Agentic Reading Roadmap. Here's the deal: dense technical books are worth reading, but almost nobody reads them cover to cover, and the ones who try usually stall out around chapter four. So instead of another summary, you get a plan: the chapters that earn their page count, the order that actually builds on itself, and permission to skip the rest. Let's get into it.
The field of AI engineering is now distinct from ML engineering. This book gives you the mental models, decision frameworks, and architectural patterns to build production-ready AI applications on top of foundation models, without training anything from scratch.
Mid-to-senior engineers building AI-powered products. Engineering managers who need to make architectural decisions (RAG vs. finetuning, API vs. self-hosted). Technical PMs and founders who are past the demo phase and need to understand why their system fails in production. ML engineers transitioning into AI engineering roles.
Researchers who want mathematical depth, there is no calculus here, and that’s intentional. Beginners who haven’t shipped anything with an LLM API yet, get your hands dirty first, then come back. Engineers who exclusively want code to run, this book has none. If you’re looking for a LangChain or LlamaIndex tutorial, this is not that book.
Basic Python. A working mental model of how APIs work. Ideally, you’ve built at least one prototype with an LLM. Familiarity with concepts like embeddings and vectors helps but isn’t required, Huyen defines everything clearly.
2 hours for the 80/20 Fast Track. 8–10 hours for the full Practitioner’s Deep Dive. The first half of the book can be read at roughly twice the speed of the second half.
If you put in the hours, here's what you walk out with:
A clear mental model of when to use prompt engineering, RAG, or finetuning and why the answer is almost never “all three at once.”
A framework for thinking about AI evaluation before you write a single line of application code. You’ll stop treating evaluation as an afterthought.
A vocabulary for latency, cost, and quality trade-offs that lets you have honest conversations with your team about what a production AI system actually needs.
A principled answer to the question: “Should we host our own model or use an API?”, based on concrete factors like goodput, KV cache utilization, and data privacy constraints, not gut feeling.
A reference architecture for an end-to-end AI engineering system: context enhancement → guardrails → model routing → caching → agents. Chapter 10 maps this out in one of the clearest system diagrams you’ll find in the field.
A decision tree for the RAG-vs-finetuning tradeoff. Huyen’s framing here is sharp: finetuning is for form, RAG is for facts.
A structured approach to designing AI evaluation pipelines, including how to use AI-as-a-judge without falling into the trap of the model grading its own homework.
Do not read this book cover-to-cover. The first four chapters are front-loaded with foundational context. Some of it is excellent; some of it is padding you do not need. Your time is better allocated elsewhere. Follow this execution path instead
The unique value of this book lives in the second half. These three chapters contain the architectural insights that take engineers months of trial-and-error to learn on their own:
Read Chapter 5 (Prompt Engineering) analytically. Not for the individual tactics (you likely know most of them), but for Huyen's framing of context construction as the central discipline of AI engineering. This reframes prompting from a hack into an engineering practice. Pay attention to the section on structured outputs and the discussion of system prompts as contracts, not suggestions.
Read Chapter 6 (RAG and Agents) in full. This is the core of the book. The RAG architecture walkthrough is the clearest available treatment of the topic, from chunking strategies through hybrid retrieval through evaluation of the retriever itself. The Agents section is equally grounded. Huyen avoids the hype trap: she covers agent failure modes, planning limitations, and why most "autonomous" agents in production are closer to structured workflows with fallback logic. Read every word here.
Read Chapter 10 (AI Engineering Architecture and User Feedback) last, before anything else. Yes, out of order. Read it first as an inspectional pass to see where the whole book is going, the reference architecture maps everything together. Then read it again after Chapter 6. The section on user feedback and conversational feedback extraction is underappreciated; it's where Huyen discusses how to actually learn from your production system rather than just monitoring it.
These chapters require active engagement. Read them at a desk, with notes open, not on a phone.
Chapter 3 and Chapter 4 (Evaluation). Arguably the most important and most underestimated chapters in the book. Huyen makes the case that evaluation is the hardest unsolved problem in AI engineering, not model selection or inference cost. The treatment of AI-as-a-judge is rigorous, she covers positional bias, verbosity bias, and self-enhancement bias in model judges, which are failure modes most teams discover the hard way. If you have an existing AI product in production with weak evaluation, read these chapters first, before anything else in the book.
Chapter 7 (Finetuning). Slow down here. The section on when not to finetune is more valuable than the section on how to finetune. Huyen walks through memory math for LoRA vs. full finetuning in a way that makes hardware constraints concrete rather than abstract. Pay particular attention to the discussion of preference finetuning (RLHF/DPO), the nuances between supervised finetuning and preference alignment are frequently confused in practice.
Chapter 9 (Inference Optimization). Dense but worth it. The framework for understanding inference bottlenecks, memory-bound vs. compute-bound regimes, KV cache behavior, quantization trade-offs, is the kind of mental model that separates engineers who can make infrastructure decisions from those who just follow benchmarks. Don’t try to memorize the numbers; focus on the reasoning structure.
Chapter 1 (Introduction): Skim the section on the AI engineering stack (the three-layer diagram is genuinely useful) and the breakdown of AI engineering vs. ML engineering vs. full-stack engineering. Skip the use-case catalogue, it reads like a product brochure and adds little if you already work in the field. Read the intro paragraph and the summary, then move on.
Chapter 2 (Understanding Foundation Models): This chapter is the most debated in reader reviews, and for good reason. The treatment of transformer architecture and training is broad but shallow. If you already understand what post-training, RLHF, and sampling strategies are, skim the section headers and skip to the Structured Outputs and Probabilistic Nature of AI subsections, those are genuinely useful framings. If you’re new to foundation models, read it analytically. If you’re experienced, you’ll lose an hour here you won’t get back.
Chapter 8 (Dataset Engineering): This chapter is excellent if dataset curation and synthetic data generation are live problems for you right now. If they’re not, meaning you’re API-first and haven’t moved to finetuning yet, skim the introductory framing and the section on data synthesis. Return to this chapter when you actually need it.
If you'd rather have this as a clean PDF you can annotate, comment "AI Engineering guide" below and I'll send you a copy, plus the supplementary resources I pulled together while writing it.
Foundation model : a large model trained on broad data that can be adapted to many tasks (distinct from task-specific ML models)
Context construction : Huyen’s term for the full discipline of deciding what to put in the prompt; treated as the central skill of AI engineering
Goodput: useful throughput in inference serving; the fraction of total compute that produces valid, complete outputs
KV cache: the key-value cache used during autoregressive decoding; understanding this is essential for Chapter 9
PEFT (Parameter-Efficient Fine-Tuning) : finetuning techniques like LoRA that update a small fraction of model weights rather than all of them
AI-as-a-judge: using a language model to evaluate the outputs of another language model; discussed at length in Chapter 3-4
Preference finetuning: training a model using human or AI preference signals rather than direct supervision (RLHF, DPO, etc.)
This is not a code book. There are no code blocks to run, no GitHub repos to clone, no environment to set up. That’s a deliberate choice and it’s both the book’s biggest strength and its most frequent criticism.
What this means for how you read it:
The “Offline Read” method applies here more than anywhere. Because there’s no code pulling your attention to a terminal, the risk is passive reading. You need to compensate deliberately. Read with a document open. After every subsection, write one sentence in your own words that captures the core trade-off or decision. If you can’t, you didn’t actually absorb it.
Build the decision tables yourself. Huyen gives you the factors; she often stops short of the decision matrix. As you read Chapters 5–7, build your own table mapping your current project to the framework she provides: Does your application need factual grounding (→ RAG)? Does it need a specific output style or format that prompting can’t achieve consistently (→ finetuning)? Making these concrete as you read is what separates reading from studying.
GitHub Archaeology: The official companion repo is at github.com/chiphuyen/aie-book. It contains chapter summaries, errata, and supplementary resources. Worth bookmarking. For code that implements the concepts, use it alongside Huyen’s own Substack and the O’Reilly learning platform exercises, the book deliberately points outward to implementations rather than including them.
The Evaluation Trap to avoid: A common mistake after reading Chapters 3-4 is trying to build a perfect evaluation framework before shipping anything. Don’t. Huyen is clear that evaluation is iterative. The value of those chapters is to know what you’re measuring and why, not to have a complete pipeline before you start.
If your Claude agent is trapped inside your app, it’s time to move the conversation to where your customers actually live. Without the headache of maintaining custom infrastructure.
Novu Connect is the new standard in Agent Communication Infrastructure (ACI). It bridges the gap between your AI agents and any messaging platform, including Slack, Teams, WhatsApp, and email. You don’t need to spend weeks building pipelines; just pick a template, connect your tools, and you’re live in under 2 minutes.
No infrastructure to babysit, no custom plumbing to rebuild. Just immediate, two-way conversations with your customers.
Join the challenge: deploy your agent in under 2 minutes and unlock a free month of our Pro tier.
Passive reading decays within 48 hours without retrieval. Here’s how you lock this in before your next meeting.
Answer these without looking at the book. If you get stuck, you found your reread target.
Huyen argues there’s a specific ordering of techniques you should try before finetuning a model. What is that ordering, and what’s the engineering rationale for it? (Chapters 5–7)
What is the fundamental reason AI-as-a-judge evaluation can fail even when the judge model is strong? Name at least two specific biases she identifies, and describe how each one distorts evaluation results. (Chapters 3–4)
In Huyen’s reference architecture in Chapter 10, what role do caches play, and why does she treat them as a mandatory step rather than a performance optimization to add later? What types of caching does she distinguish between?
Pick one AI application you’re currently building or have recently shipped, even a prototype counts.
Map it against Huyen’s five-step reference architecture from Chapter 10: context enhancement, guardrails, model routing and gateway, caching, and agent patterns. For each step, write one sentence on whether you’ve implemented it, skipped it intentionally, or skipped it by default (meaning you didn’t think about it).
Then identify the one gap in your current architecture that is most likely causing inconsistent output quality or latency spikes. Write a one-paragraph proposal for what you’d add or change, grounded in Huyen’s framework, not a tool recommendation, but a structural decision.
That paragraph is the deliverable. Share it in the comments if you want a gut-check from the community.
Production AI needs to connect evidence across sources, reason through relationships, and remember what it has already discovered.
Join the live, hands-on Building Intelligent AI Agents with GraphRAG workshop led by David Knickerbocker, and learn how to build agents that run multi-step investigations, maintain investigation memory, answer follow-up questions, and use graph, timeline, sentiment, and geospatial tools for deeper insights.
Move beyond basic retrieval and start building AI systems designed for complex, real-world questions, guided by an expert at the forefront of GraphRAG innovation.
As an AI Agents Simplified subscriber, you can get 40% off with code AIAGENTS40.
Seats are limited. Secure yours before the offer ends.
You’ll also receive:
One complimentary Packt eBook worth $35
A Packt Certificate
Recording access
Join us and learn how to build AI agents that do more than retrieve documents.
Have you read this one? Reply and tell me which section hit different or which part you think hasn’t aged well given how fast the field moves. The RAG chapter in particular is worth a conversation.
Hey there, I’m Hana, co-founder and technical writer at AI Agents Simplified. If you’re building in the AI space, have feedback on today’s post, or want to explore a collaboration, let’s chat! You can find me on LinkedIn or drop me an email. I read every message and would love to hear from you.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.