Generative AI Engineering Interview 2026

Preview: link

Book link: Guide that will help you get hired

What a generative model actually estimates, why the Transformer won, and the three numbers—tokens, parameters, and context—that price every system you will ever design on top of one.

Q1.1 Explain the difference between discriminative and generative models, and describe a production system where you would deliberately use both.

Q1.2 Your team estimated context costs using characters divided by four. After launching in Japan, users report truncated answers and your bill is 2.5x forecast. Diagnose and fix.

Q1.3 Why does the attention formula divide by √d_k? What breaks if you remove it?

Q1.4 Compare Pre-LN and Post-LN Transformers. Which would you pick for a 70B model and why?

Q1.5 A stakeholder asks: “Models now have 1M-token context—can we delete our RAG pipeline?” Give your recommendation with reasoning.

Q1.6 Explain scaling laws and Chinchilla optimality. Your company can spend a fixed compute budget—how do you allocate it, and does the answer change if the model will serve production traffic for two years?

Q1.7 What are emergent abilities, and how would you respond to a colleague who claims they prove models will suddenly become dangerous at some scale?

Q1.8 Your RAG system returns semantically similar but factually wrong passages. A colleague proposes a better embedding model. Is that the right fix?

Q1.9 Walk through what happens between a user pressing Enter and the first token appearing on screen.

Q1.10 Design the model-selection strategy for a startup with three workloads: high-volume classification, customer-facing chat, and a low-volume legal analysis tool.

Every architectural choice in a modern LLM—RoPE, GQA, MoE, FlashAttention—is an answer to a memory or bandwidth problem, not an intelligence problem. This chapter teaches you to read architecture as a systems engineering problem.

Q2.1 Derive the KV cache size for Llama-3-70B at 8k context and explain what it means for your serving architecture.

Q2.2 Explain MHA, MQA, GQA, and MLA. Which would you choose for a 30B model serving 5k concurrent chat users?

Q2.3 Your team fine-tuned a model with RoPE base scaling to extend context from 8k to 64k. Benchmarks look fine, but users report the model “forgets” material in long documents. Debug this.

Q2.4 Why is FlashAttention faster if it performs the same number of FLOPs?

Q2.5 A colleague proposes switching from a dense 70B model to a 400B-total / 40B-active MoE to “get better quality at the same cost.” Evaluate this.

Q2.6 Explain the KV cache to a backend engineer with no ML background, then explain why it causes out-of-memory errors under load.

Q2.7 Why are LLM outputs non-deterministic even at temperature 0? Your legal client requires reproducibility—what do you tell them?

Q2.8 Design the attention configuration for a code-completion model that must serve a 50ms p99 latency with a 32k context.

Q2.9 What is the difference between the prefill and decode phases, and why does it dictate your entire serving architecture?

Q2.10 You must serve a 7B model on a single 24GB consumer GPU with 8k context and maximum concurrency. Walk through your memory budget and optimisations.

A prompt is not a magic incantation. It is the only API surface you have to a model whose weights you cannot change—so treat it like an API: versioned, tested, typed, and defended against hostile input.

Q3.1 Your prompt works in testing and fails in production 15% of the time. Walk through your debugging process.

Q3.2 Explain prompt injection and design a defence for a RAG system that ingests customer-uploaded PDFs.

Q3.3 When should you use few-shot prompting versus fine-tuning? Give the decision criteria and the crossover math.

Q3.4 A support bot leaks its system prompt when users ask cleverly. How serious is this and what do you do?

Q3.5 Design a prompt system for a multi-tenant SaaS where each customer needs custom behaviour but you must control safety and cost.

Q3.6 Explain chain-of-thought prompting. When does it hurt, and how would you decide whether to use it?

Q3.7 Your extraction pipeline returns malformed JSON about 3% of the time, and you have a retry loop. Critique this design.

Q3.8 Design an A/B testing framework for prompt changes in production.

Q3.9 How do you handle prompts that must work across multiple model providers?

Q3.10 A prompt that worked for months suddenly degrades after a provider model update. Walk through your response.

Training is a research problem you solve once. Serving is an engineering problem you solve every day, at every traffic level, under a latency SLO—and it is where most of the money goes.

Q4.1 Explain continuous batching and quantify the improvement over static batching.

Q4.2 Your service has a p50 TTFT of 200ms but a p99 of 8 seconds. Diagnose.

Q4.3 Explain PagedAttention and why it was a breakthrough.

Q4.4 What is speculative decoding, when does it help, and when does it hurt?

Q4.5 Design an LLM serving system for 10,000 requests/second with mixed workloads.

Q4.6 Your inference cost is $180k/month. The CFO wants it halved without any loss of quality. What do you do?

Q4.7 How do you handle streaming responses, and what breaks in production?

Q4.8 Compare vLLM, TensorRT-LLM, and llama.cpp. When would you choose each?

Q4.9 Your GPU fleet OOMs during traffic spikes despite 30% headroom at steady state. Explain and fix.

Q4.10 Explain the memory-bandwidth bound in decode and what actually helps.

RAG is not “add a vector database.” It is an information retrieval system with an attached language model, and almost every RAG failure in production is an IR failure that people try to fix with prompting.

Q5.1 Design a RAG system for 50 million enterprise documents with per-user access control.

Q5.2 Users say your RAG chatbot gives correct answers to the first questions but nonsense on follow-ups. Diagnose.

Q5.3 Explain the difference between bi-encoders and cross-encoders and why production systems use both.

Q5.4 Your RAG system’s answers are technically grounded, but users say they “miss the point.” Investigate.

Q5.5 How would you evaluate a RAG system before and after launch?

Q5.6 Compare chunking strategies. How would you choose a corpus of legal contracts?

Q5.7 Your retrieval latency is 800ms p99, and the budget is 200ms. Optimise.

Q5.8 When would you NOT use RAG?

Q5.9 Design a RAG evaluation and monitoring system that detects quality degradation before users complain.

Q5.10 You must migrate a 200-million-chunk index to a new embedding model with zero downtime. Design the migration.

An agent is a loop with a budget, a memory, and the ability to act. Everything that makes agents hard—error compounding, cost variance, security blast radius—follows from those three properties.

Q6.1 Design an agent that resolves customer support tickets end to end. Cover architecture, tools, safety, and evaluation.

Q6.2 Your agent works in testing but loops infinitely in production. Debug it.

Q6.3 When should you use a multi-agent architecture versus a single agent with more tools?

Q6.4 How do you handle tool call failures and make agents resilient?

Q6.5 An agent with database write access is deployed. What are the security risks and how do you mitigate them?

Q6.6 How do you evaluate an agent? Design the eval suite for a coding agent.

Q6.7 Your agent’s average cost is $0.30 per task, but p99 is $12. Explain and fix.

Q6.8 Design the memory architecture for a personal assistant agent used daily for years.

Q6.9 How would you migrate a rule-based workflow automation system to an agentic one? What stays rules-based?

Q6.10 An agent in production suddenly starts failing after a model provider update. Walk through the response.

Every modality is converted into tokens before a language model can reason over it. Knowing what that conversion costs—in tokens, in information, and in latency—separates a multimodal engineer from someone calling a vision API.

Q7.1 Design a document understanding system for insurance claims: scanned forms, photos of damage, and handwritten notes.

Q7.2 Your VLM performs well on benchmarks but fails on your customers’ documents. Diagnose.

Q7.3 Explain how vision-language models work. What is the connector, and why does it matter?

Q7.4 Design a real-time voice assistant with under 500ms response latency.

Q7.5 Multimodal RAG over technical manuals with diagrams. Users ask questions, and the diagrams answer. How do you build it?

Q7.6 Your VLM-based OCR system reports 96% accuracy, but finance says the numbers are wrong. Investigate.

Q7.7 How do you handle video understanding when a 10-minute video exceeds any context window?

Q7.8 Compare a specialized OCR pipeline against an end-to-end VLM for invoice processing at 100,000 documents/month.

Q7.9 Explain diffusion models and classifier-free guidance. How would you productionise image generation at scale?

Q7.10 Design content moderation for a user-facing image generation product.

Fine-tuning reliably teaches a model how to behave. It teaches it what is true badly, expensively, and un-updatably. Almost every disappointing fine-tune traces back to confusing those two.

Q8.1 When should you fine-tune versus use RAG? Give a decision framework and a case where you need both.

Q8.2 Explain LoRA mathematically. Why does it work, and what are the practical hyperparameters?

Q8.3 Your fine-tuned model is better at the target task but worse at everything else. Explain and fix.

Q8.4 Design a fine-tuning pipeline for a model that must be updated monthly with new data.

Q8.5 Compare RLHF, DPO, and KTO. Which would you choose for a product with a thumbs up/down button?

Q8.6 You have 500 hand-labeled examples and need 10,000. How do you generate synthetic data safely?

Q8.7 How do you serve 50 fine-tuned model variants for 50 customers cost-effectively?

Q8.8 Your fine-tune shows 15% improvement on your eval set, but users report no difference. What went wrong?

Q8.9 Your SFT run’s training loss is decreasing smoothly, but eval quality is flat or worse. Debug it.

Q8.10 How would you distill a frontier model into a 3B model you can serve yourself? Cover data, method, legality, and evaluation.

The model is the easy part. Everything that turns a working prototype into a service someone will pay for—guardrails, caching, quotas, observability, deployment—is ordinary distributed systems engineering applied to an unusually expensive and unusually unpredictable dependency.

Q9.1 Design the complete production architecture for an enterprise AI assistant serving 50,000 employees.

Q9.2 Your LLM costs jumped 300% overnight with no traffic increase. Investigate.

Q9.3 Design a guardrail system. What do you check, where, and how do you handle guardrail failures?

Q9.4 How do you monitor LLM quality in production when there is no ground truth?

Q9.5 Design a semantic caching system. What are the risks and how do you mitigate them?

Q9.6 A provider outage takes down your main model. Design the failover strategy.

Q9.7 How do you handle PII in an LLM pipeline that must comply with GDPR?

Q9.8 Design the deployment and autoscaling strategy for self-hosted LLM inference on Kubernetes.

Q9.9 A customer reports the model “stops mid-sentence” intermittently. Walk through the investigation.

Q9.10 Design the on-call runbook for a generative AI service. What alerts fire, and what does the responder do?

You cannot improve what you cannot measure, you cannot afford what you cannot price, and you cannot pass a senior interview without being able to design the whole system on a whiteboard. This chapter covers all three.

Q10.1 Design an end-to-end evaluation system for a customer-facing LLM product from scratch.

Q10.2 Design a production RAG system for a legal firm: 10 million documents, strict accuracy requirements, full auditability.

Q10.3 How do you detect and reduce hallucinations in a production RAG system?

Q10.4 Your CFO wants to cut AI spend 50% without hurting quality. Present your plan.

Q10.5 Design an AI system that summarises 10,000 documents daily with 99.9% reliability.

Q10.6 Compare LLM-as-judge with human evaluation. When is each appropriate, and how do you validate a judge?

Q10.7 Walk me through designing an AI coding assistant end-to-end.

Q10.8 What are the most important lessons you would give someone building their first production generative AI system?

Q10.9 You are launching a new product with no production traffic. How do you build an evaluation set from a cold start?

Q10.10 Design the process for deciding whether to deploy a quantized model. FP16, FP8, INT8, or INT4?

Apply coupon code below 100% FREE 👇👇👇