Generative AI Development · part of The Exam Room The situation The subscriber help desk agent has to reach four things. An internal billing API, to check whether a charge was correct. An internal delivery API, to read the schedule for a postcode. A third-party routing service, which authenticates with an API key. And, for the subscribers who opted into it, their calendar, so the assistant can…
Generative AI Development · part of The Exam Room The situation The subscriber help desk runs a reasoning loop the team wrote themselves, hosted on the AgentCore runtime. That decision is settled: they wanted control over the prompt structure, the tool-calling contract, and which model answers each step, and the harness would have taken all three. Now a second agent is coming. Operations want one…
Generative AI Development · part of The Exam Room The situation A logistics company has three model-backed features converging on the same release train. The customer-facing assistant calls a hosted model that AWS operates; its traffic is spiky and daytime-shaped, near zero overnight. A document classifier is midway through a training run on the team’s own labelled data, due in about three weeks.…
The Corner Room books live music five nights a week: a couple of headline acts, a residency, an open-mic night, a DJ on Fridays. Every gig deserves a poster, an image for the listing page, and a short loop for the socials, and for years the venue has managed exactly one of those, sometimes, when the booker’s flatmate had an evening free. The listings database, meanwhile, knows every fact a poster…
Part of The Right Tool · The Greenbox Story Sam catches it on a Thursday afternoon. She’s in the staging environment because Kai mentioned at standup that the new payment debugging tool was ready to test. Sam isn’t a developer. But she’s been handling payment support tickets for eighteen months, and when someone says “the debugging tool is ready,” Sam is the person who actually tries to debug a…
Event storming builds the model; this session is how the model survives contact with the people who know the domain. The wall on the screen is generated from a discovery ledger, every open question is a red card sitting beside its impact, and the session’s score is the red count going down. Walking the Wall Walking the Wall is a facilitated review of an existing event-storm model, run against a…
Part of The Service Manual series · Under the Hood Amazon S3 turned twenty in March 2026. It stores more than 500 trillion objects, answers over 200 million requests a second, and nearly everything else AWS sells is built on top of it. The API is small enough to learn in an afternoon; the machine behind it is one of the largest distributed systems ever operated, and its internals are unusually…
Generative AI Development · part of The Exam Room Q. Keep customer PII out of prompts and logs. What is the built-in control? A. A Bedrock Guardrail with a sensitive-information (PII) policy can block or mask PII in inputs and outputs, and redacting before logging keeps it out of the invocation logs. Encrypt the log store with a customer-managed KMS key and lock down access. Why? The log that…
Build it, buy it, or borrow it? Wardley Mapping shows which capabilities differentiate you and which are commodity, so you stop building things AWS already sells for the price of a coffee. Worked example: Build, Buy, or Borrow? . Wardley Mapping Wardley Mapping plots the components that serve a user need against two axes, visibility to the user (vertical) and stage of evolution from novel to…
Generative AI Development · part of The Exam Room Last-pass revision for securing and governing generative AI on AWS. Skim the tables, drill the traps. Controls at a glance Concern Control Notes Identity IAM identity-based policy scoped to model ARNs bedrock:InvokeModel on a specific foundation-model/* ARN, not * ; roles, not long-lived keys Identity (enablement) Model access in the Bedrock…
Generative AI Development · part of The Exam Room This starts the managed track of the hands-on labs. The first ten build everything by hand against the model API. These two use a managed service instead, and the point of going second is that you already know what the service is doing for you. The from-scratch lab made you write embed-compare-rank yourself; this one hands the same Greenbox…
Part of The Right Tool · The Greenbox Story Anika sends Charlotte a message on a Wednesday morning. It’s long, by Anika’s standards. “The Melbourne squad just spent 25 minutes Example Mapping ‘update subscriber email address.’ No red cards. Two examples. One rule: the email has to be valid. I watched five adults sit in a room with coloured cards to collectively arrive at the conclusion that an…
Generative AI Development · part of The Exam Room The situation A media and operations team has landed two projects in the same sprint, and because both say the word “images” in the brief, someone has filed them under one ticket. The first project is a marketing pipeline: given a product name and a short brief, produce on-brand hero images and a few seconds of promotional video, at volume, without…
Generative AI Development · part of The Exam Room Q. What gives you operational visibility into a production Bedrock app? A. CloudWatch metrics (invocation counts, latency, throttling, token usage) and alarms, plus model invocation logging for prompt and response inspection. CloudTrail covers configuration changes. Why? Ops metrics and content records answer different questions; you want both.
Generative AI Development · part of The Exam Room The situation An engineering-docs team is building a retrieval assistant over a large internal corpus on Amazon Bedrock. The source material is not clean prose. It is runbooks and API references full of code blocks, architecture docs with wide comparison tables, and onboarding pages that mix headings, bullet lists, sample payloads, and screenshots…
Generative AI Development · part of The Exam Room The situation A team has a working generative-AI feature. It is a support assistant built on Amazon Bedrock: a customer types a question, the app retrieves a few relevant policy documents, stuffs them into a prompt alongside a Claude model, and returns an answer. In a notebook, driven by hand, it is genuinely impressive. The retrieval pulls the…
Generative AI Development · part of The Exam Room The situation A team is putting a generative-AI assistant into production. It answers questions, summarises documents, and drafts responses, all on Amazon Bedrock. Legal, security, and the product owner each hand over a list of things the assistant must never do, and the lists do not look alike. Some entries are the usual suspects: no hate speech,…
Generative AI Development · part of The Exam Room A fast revision pass over evaluating, monitoring, costing, and operating a generative AI app on Bedrock. Skim the tables, drill the decision rules, watch the traps. Levers at a glance Concern Tool / lever Notes Quality baseline Golden set Fixed prompt/answer pairs including hard and out-of-scope cases; the yardstick every change is measured against…
Generative AI Development · part of The Exam Room This is the final hands-on lab. The scaffolding is gone. You get a requirement and a test, and you write the whole handler. The full lab is in lab-10-capstone.zip . Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper , a standing backstop that…
Generative AI Development · part of The Exam Room The situation A team is shipping an internal assistant on Amazon Bedrock that answers staff questions about HR policy, expenses, and IT access. It runs a retrieval step first, pulling the relevant policy passages from a knowledge base, then hands the model the question and the passages. It also has two tools: one that looks up an employee’s…
Generative AI Development · part of The Exam Room This is the second lab in the managed track of the hands-on labs. The first ten build things by hand against the model API; the managed track lets AWS do it instead. The theory post on tuning fine-tuning laid out the knobs and what a training-versus-validation loss curve looks like when a run goes wrong. Nothing on this blog has actually run one.…
Generative AI Development · part of The Exam Room The situation A team runs a documentation assistant on Amazon Bedrock. A user asks a question, the app embeds it, queries a vector store for the most similar chunks, stuffs the top matches into a prompt alongside the question and a block of standing instructions, and sends the whole thing to a Claude model for the answer. The corpus is a few…
Part of Under the Hood — deep dives into the technology we use every day. In 2005, Hurricane Katrina slammed into the Gulf Coast of the United States. In the chaos that followed (flooded streets, collapsed infrastructure, overwhelmed emergency services) one of the most effective response tools was a simple mashup. Volunteers took NOAA’s freely available satellite imagery and overlaid it with…
Generative AI Development · part of The Exam Room The situation A team has built a retrieval-augmented feature on Amazon Bedrock. Documents are chunked, run through an embedding model to produce vectors, and stored in an OpenSearch k-NN Nearest-neighbour search Finding the vectors closest to a query vector; at scale it’s approximated, trading a little accuracy for a lot of speed. index; at query…
Generative AI Development · part of The Exam Room Q. Your Bedrock bill is high but quality must hold. First levers? A. Right-size the model per task (a smaller model where it suffices), cache repeated responses and context, trim prompt and output tokens, and batch where latency allows. Reserve provisioned throughput Provisioned Throughput Reserved Bedrock capacity bought by the hour for a fixed…
Generative AI Development · part of The Exam Room The situation A team is running a customer-facing chat assistant on Amazon Bedrock. Each user turn is a stateless model call: the runtime holds no memory between requests, so the application has to gather the running transcript, any session facts (the user’s plan, the current basket, what the assistant already asked), and hand the whole context…
The hardest scaling problem most growing companies face isn’t technical; it’s knowledge. At five people, the founder can answer every question about the business. At twenty-five people across three cities, they can’t. Underneath every healthy scaling story is the same pattern: knowledge moving from one person’s head into containers that the rest of the team can use without that person in the room.…
Generative AI Development · part of The Exam Room The situation A product team has a backlog of features that all sound like AI work, and the default plan for every one of them is to call a large language model on Amazon Bedrock. There is a form that needs a postcode validated. There is a stream of scanned invoices to pull totals out of. There is a support inbox that needs each message routed to…
Generative AI Development · part of The Exam Room The situation An internal assistant has been rolled out across a mid-sized company. It runs on Amazon Bedrock, answers questions from an HR and finance knowledge base built over policy documents, past tickets, and spreadsheets, and can call a few tools: one that looks up an employee record, one that pulls a team’s expense summary, one that drafts a…
Generative AI Development · part of The Exam Room A dense pass over how you push a foundation model closer to your task, from cheapest to heaviest, and how it gets served once you have. The ladder at a glance Approach Changes what Needs Serve via Prompt engineering Nothing in the model; only the input A good prompt, few-shot examples, system instructions Base model, on-demand RAG Nothing in the…
Generative AI Development · part of The Exam Room This is one of the hands-on labs that run alongside these posts. The scaffolding is nearly gone: the harness is here, the judgement is yours to design. The full lab is in lab-09-evaluation.zip . Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper , a…
Generative AI Development · part of The Exam Room The situation A retail company runs a customer-service assistant on Amazon Bedrock, backed by a knowledge base for policies and a set of tools that read the signed-in customer’s account: orders, subscriptions, addresses, payment methods. It handles returns, order status, plan changes, and general policy questions. Most days it works well. The…
Part of The Right Tool · The Greenbox Story Greenbox has 5,800 subscribers. Three squads. Twenty-five people. Two cities, with Brisbane on the way. And a substitution engine that’s about to get a lot more complicated. The Perth squad has picked up a major upgrade: seasonal rules, allergen combinations, and subscriber preference learning. It’s the most complex code in the system, and it touches…
Generative AI Development · part of The Exam Room The situation A team is building a retrieval-augmented support assistant over a documentation corpus: product manuals, release notes, an internal knowledge base, and a few thousand resolved support tickets. Queries come from two very different mouths. End users ask things in natural language, “why does my box arrive warm”, and internal agents paste…
Generative AI Development · part of The Exam Room This is the third lab in the managed track. The fine-tuning lab taught the model to sound like Greenbox; this one teaches the pipeline to draw like it. It is also the track’s first trip out of text: image generation, video generation, and the asynchronous invocation pattern that video forces on you. The full lab is in lab-13-weekly-creative.zip ;…
Generative AI Development · part of The Exam Room The situation A knowledge-base assistant on Amazon Bedrock retrieves passages to ground its answers. The embeddings live in a vector store, and at launch the corpus was 40,000 chunks Chunking Splitting documents into retrievable pieces before embedding them – small enough to match precisely, big enough to still make sense. . Queries came back in a…
Generative AI Development · part of The Exam Room Q. Thirty services share prompts and you need versioning and reuse. What on Bedrock? A. Bedrock Prompt Management stores, versions, and shares prompts as managed resources, so a service references a version instead of copy-pasting prompt text. Prompt caching Prompt caching Reusing the model’s already-processed prefix (system instructions, fixed…
Generative AI Development · part of The Exam Room The situation A product team runs a customer-facing assistant on Amazon Bedrock. It has a system prompt, a set of few-shot examples, a guardrail that blocks unsafe topics and redacts PII, and it calls a couple of internal tools through a Bedrock agent. All of it is wired together in application code: the model is referenced by a convenient alias,…
Generative AI Development · part of The Exam Room The situation A company has standardised on Amazon Bedrock and the demand is now organisation-wide. A dozen product teams, spread across separate AWS accounts under one AWS Organization, all want to invoke foundation models. Some teams have a genuine need for the most capable and most expensive models; most do not. One team handles regulated data…
Generative AI Development · part of The Exam Room The situation A knowledge-assistant team has shipped a Bedrock-backed feature that answers staff questions over an internal document set. A request retrieves passages from a vector store, stuffs them into a prompt with the conversation history, calls the model, and sometimes makes a tool call to look up a live figure before answering. It works, and…
Generative AI Development · part of The Exam Room Fast revision for choosing between model-decided and deterministic control flow, and the Bedrock services that back each one. Options at a glance Approach Control flow Use when Plain Converse call You, in code Single prompt in, single answer out; no tools, no loop Converse loop with tool use Model picks the tool, you run it Model needs live data or…
Generative AI Development · part of The Exam Room This is one of the hands-on labs that run alongside these posts. The full lab, with the database and the read-only guard, is in lab-08-text-to-sql.zip . Before your first lab, do the one-time, once-per-account setup: run the zip’s preflight.sh to confirm your account is ready, then deploy the lab reaper , a standing backstop that auto-deletes any…
Generative AI Development · part of The Exam Room The situation A retailer runs a customer-service assistant on Amazon Bedrock. It answers subscription questions, checks order status through a couple of internal tools, explains the returns policy, and updates delivery preferences. For its first few thousand conversations it did all of that well, and the team was pleased with how rarely it needed a…
Generative AI Development · part of The Exam Room The situation A retailer is building a customer assistant on Amazon Bedrock. It has to do three quite different jobs from one chat surface. It answers policy questions (“how long do I have to return an item?”), which live in a few hundred pages of help-centre articles and terms documents that change a handful of times a year. It answers order…
Part of the Trust series · Under the Hood This is the final post in the Trust series. We’ve covered the fundamental problem , identity , encryption , and certificates , all of which are mechanisms for establishing trust between machines, or between a machine and a person. But what about trust between people who’ve never met? You can’t demand a cryptographic certificate from an eBay seller. You…
Generative AI Development · part of The Exam Room The situation A team has a working retrieval-augmented-generation assistant over a knowledge base of a few thousand support articles and product docs. The documents are chunked, embedded, and stored in a vector index; at query time the retriever pulls the nearest chunks Chunking Splitting documents into retrievable pieces before embedding them –…
Generative AI Development · part of The Exam Room Q. Steady high-volume Bedrock traffic with latency guarantees. Provisioned Throughput or on-demand? A. Provisioned Throughput: reserved model units Model unit The billing block Provisioned Throughput is sold in – one unit delivers a fixed tokens-per-minute rate for a specific model. for predictable, high, steady load and consistent latency (and the…
Generative AI Development · part of The Exam Room The situation A product team has two generative-AI features heading for production. The first is a customer-facing assistant that answers billing and account questions in natural language, spiky traffic that peaks during business hours and goes quiet overnight. The second is a batch job that runs every night over a large backlog of long case files,…
Generative AI Development · part of The Exam Room The situation A product team has shipped an AI reply drafter on Amazon Bedrock. It reads a customer support thread, pulls a few relevant help-centre articles through retrieval, and drafts a response the agent can edit and send. It has been live for two months, thousands of drafts a day, and the team has a thumbs up and thumbs down button under each…
Generative AI Development · part of The Exam Room The situation A platform team maintains a mid-sized codebase: a few dozen services, a shared internal SDK, and a house style that every new file is expected to follow. They want to add code assistance in several places. Developers want in-editor completions and a chat window that knows the repo. A migration project needs a service translated from…