Lesson 1: Architecture — Designing the Five-Component Pipeline

Today we build:

• The five-stage contract every later phase of this course plugs into

• A working, testable pipeline skeleton — no real model calls required

• The one architectural decision that determines whether you can add evals, audits, and cost controls later without a rewrite

Most AI features start as one function: take a question, call a model, return the answer. It works in the demo. It becomes unmaintainable the first time something goes wrong, because there’s no place to insert a check, log a decision, or measure a cost — everything happens inside one opaque call. Today’s architecture decision is the one choice you can’t easily undo later, so we make it deliberately, on Day 1, before writing a line of logic.

A single-call AI feature has exactly one seam: the API call itself. If the output is wrong, you have one place to look and no way to tell whether the problem was understanding the question, finding the right information, or writing the answer. You’re debugging a black box.

The fix isn’t a bigger model — it’s more seams. We split the work into five stages, each with a narrow, explicit responsibility: Query Understanding turns a raw question into a structured intent. Retrieval finds the information needed to answer it. Synthesis drafts an answer from that information. A Critic checks the draft against the original intent before anyone sees it. Formatting shapes the final response for its destination — API, chat window, or app.

Five stages sounds like more surface area to break. It’s the opposite. Each seam is a place where next week’s eval harness can score an output, where a future audit log can record a decision, and where a cost tracker can attribute a dollar. A monolithic call gives you none of that. The stages don’t make the system more complex — they make the complexity visible, one boundary at a time.

The rule for today: every stage takes a typed input and returns a typed output. Nothing passes a raw string or a loose dictionary between stages. That discipline is what lets Phase 1 through Phase 5 attach reliability tooling to this pipeline without touching the pipeline itself.

Data flows in one direction: a Query enters, and a FinalResponse leaves. Each stage owns one transformation and knows nothing about the stages before or after it beyond the shape of the data it receives and returns. The Pipeline object is the only piece that knows the full sequence — swap any one stage’s implementation later, and nothing else needs to change.

https://github.com/sysdr/ai-reliability-engineering/tree/main/lesson_01_package%20(1)/lesson_01_package

Start by defining the data contracts — these are the types every stage will pass along:

Each stage is a class with one method — run() — that takes the previous stage’s output type and returns its own. Today, each stage’s internal logic is intentionally simple (deterministic, no API calls) — you’re proving the seams work before you make any single stage smarter. The Pipeline class calls all five in sequence and returns the final response.

Everything — install, build, run, test, verify — is one command:

start.sh installs dependencies, builds the Docker image if Docker is available (skips gracefully if not), runs the pipeline, runs the tests, and prints a verification summary:

Query received: "What's the refund policy for annual plans?"
Stage 1 (Query Understanding) -> intent: refund_policy
Stage 2 (Retrieval) -> 2 passages found
Stage 3 (Synthesis) -> draft answer generated
Stage 4 (Critic) -> approved: True
Stage 5 (Formatting) -> response ready
Final: "Annual plans can be refunded within 30 days of purchase."
== Running tests ==
4 passed

When you’re done, clean everything up:

lesson_01_package/
├── lesson_code.py    # the five stages + Pipeline class
├── test_lesson.py    # 4 tests verifying the pipeline is wired correctly
├── requirements.txt  # pytest only — no API keys needed today
├── Dockerfile         # minimal python:3.11-slim image
├── start.sh           # install deps, build, run, test, verify — one command
├── stop.sh            # cleanup — removes .venv, caches, and the Docker image
└── README.md          # quick-start reference

start.sh does five things in order: creates a virtual environment, installs requirements.txt, builds the Docker image if Docker is present (skips cleanly if not), runs lesson_code.py, then runs test_lesson.py. You should see 5 stage log lines, a final answer, and 4 passed.

When you’re done: ./stop.sh — removes the venv, __pycache__, .pytest_cache, and the Docker image if one was built.

Five typed dataclasses define every handoff between stages:

From To Type caller Query Understanding Query (text, session_id) Query Understanding Retrieval Intent (query, intent_label) Retrieval Synthesis RetrievedContext (intent, passages) Synthesis Critic DraftAnswer (context, text) Critic Formatting CriticVerdict (draft, approved, reason) Formatting caller FinalResponse (text, approved)

Each stage class exposes exactly one public method, run(), that takes the previous type and returns the next. The Pipeline class is the only object that knows the full call order — this is what makes it possible to replace, say, RetrievalStage in Day 2 without touching SynthesisStage, CriticStage, Pipeline, or any test outside test_lesson.py.

• QueryUnderstandingStage — classifies intent with a simple keyword match today. This is the seam Phase 1’s eval harness will score against a golden dataset of real questions.

• RetrievalStage — looks up from a hardcoded dict today. Day 2 replaces the internals with real ingestion and search; the run(intent) -> RetrievedContext contract doesn’t change.

• SynthesisStage — picks the first matching passage today. This becomes the seam where Phase 1’s LLM-as-judge later scores answer quality.

• CriticStage — the seam Phase 1’s CI gate eventually wraps: nothing ships past this point without passing a check.

• FormattingStage — the seam Phase 3’s audit trail will log from — every FinalResponse becomes a queryable, replayable event.

• ModuleNotFoundError: No module named 'lesson_code' — run test_lesson.py from inside the lesson_01_package directory, not from a parent folder.

• Docker build step skipped — expected if Docker isn’t installed locally; start.sh falls back to a plain Python run automatically.

• Python version errors — this lesson uses list[str] generic syntax, which requires Python 3.9+; the Dockerfile pins 3.11 to avoid this entirely.

This pattern is why a well-run support-escalation system can tell you exactly which step misfired when a customer gets a wrong answer — was it a retrieval miss, a synthesis error, or a critic that should have caught it? Systems built as one opaque call can only tell you that something, somewhere, went wrong. That difference is the entire reason this course spends 90 days on it instead of shipping the one-call version and hoping.

Day 2, we build the first real stage: ingestion. You’ll take unstructured source documents and turn them into the retrievable units that Stage 2 will search over — the raw material the rest of the pipeline depends on.