Lesson 5 — Security Hardening

This lesson adds an output-side security layer to the AI service. It scans generated text for leaked secrets and PII before responses leave the system, and signs payloads with HMAC so integrity can be verified downstream. Findings are redacted, counted, and exposed through a live dashboard. The result is a self-contained service that enforces rather than observes.

• Regex-based detection of secrets (API keys, tokens) and PII (email, phone, card).

• Deterministic redaction with per-category counters.

• HMAC-SHA256 signing and constant-time verification.

• FastAPI dashboard with non-zero, demo-driven metrics.

• A security domain package (hardening, metrics, service).

• An OutputGuard that scans and redacts text.

• A SignatureVerifier for HMAC signing and tamper detection.

• API surface (/scan, /sign, /verify, /metrics, /demo, /dashboard, /health).

• Unit and API tests plus Docker lifecycle scripts.

The previous lesson enforced throughput and cost limits before model execution. This lesson extends enforcement to the output boundary, guarding what the system emits rather than what it accepts.

Clean, signed, redacted output is a precondition for trustworthy telemetry, enabling the observability stack in the next lesson to reason over safe data.

• Where it sits: at the response egress, between model output and the caller.

• Why it exists: models can echo secrets or personal data present in prompts, context, or tool results.

• Problem solved: prevents data exfiltration and provides integrity guarantees on payloads crossing trust boundaries.

FastAPI validates payloads with Pydantic and delegates to SecurityService. The service runs the guard and signer, records outcomes in a shared in-memory metrics store, and returns a decision. The dashboard polls /metrics and triggers /demo.

The module targets three production objectives: prevent sensitive-data leakage, guarantee payload integrity, and make both measurable for operators.

Output scanning is pattern-driven because secrets and PII share stable syntactic shapes. Compiling patterns once and applying them per request keeps latency low while covering multiple categories in a single pass. Detection and redaction are separated conceptually but computed together: detection drives metrics and block decisions, redaction produces a safe-to-emit string.

HMAC signing solves a different problem — integrity, not confidentiality. A shared secret produces a signature that a holder can verify but an attacker cannot forge without the key. Verification uses constant-time comparison to avoid timing side channels.

A deliberate design choice treats an expected rejection (a tampered payload correctly failing verification) as a success signal, while an unexpected accept is the true failure. This aligns metrics with security intent rather than raw booleans. State is kept in-process to keep scope small while preserving interfaces that can move to shared stores later.

This lesson is the egress guardrail of an AI platform. It belongs next to response serialization and gateway middleware, where every model output can be inspected, redacted, and attested before delivery.

• Request flow: caller submits text to scan or a payload to sign/verify.

• Execution flow: the service runs guard or signer, then records an event.

• Data flow: validated input enters the service, mutates in-memory metrics, and returns a decision; the dashboard reads snapshots via /metrics.

• State changes: scan/block counters increment, secret/PII tallies grow, HMAC counters update, and a bounded recent-events list is refreshed.

• Architecture fit: embed as egress middleware in API workers.

• Enterprise patterns: centralize pattern sets and signing keys per tenant or environment.

• Scalability: stateless scanning scales horizontally; shared counters move to Redis when multi-worker.

• Observability: counters, rates, and an event timeline feed dashboards and alerts.

• Security: secrets come from environment variables; comparisons are constant-time.

https://github.com/sysdr/production-ai-engineering/tree/main/lesson5/aiam-day05

The guard compiles patterns at construction. Each scan collects findings, computes category counts, and builds a redacted copy. The service wraps guard and signer, timing each call and writing an event to the metrics store. The API layer validates inputs, exposes endpoints, and serves the dashboard. Errors surface as validation failures at the edge. The demo exercises clean text, every detector, and both HMAC paths so no metric stays zero.

Constant-time verification prevents timing attacks:

Intent-aware status keeps metrics meaningful — a caught tamper is a win, not a failure:

This matters because naive boolean logging flags correct security behavior as an error.

• Scalability: externalize counters for multi-worker deployments.

• Security: never log raw findings; store only categories.

• Monitoring/logging: track block rate and HMAC failure rate.

• Testing: cover each detector and both verification paths.

• Failure handling: fail closed on scan errors.

• Edge cases: overlapping patterns and unicode payloads.

• Verification: unit tests assert detection, redaction, and HMAC correctness.

• Testing strategy: API tests confirm endpoints and non-zero demo metrics.

• Success criteria: all tests pass; dashboard values update on demo.

• Expected outputs: blocked secrets/PII, valid/tampered HMAC results.

• Benchmarks: sub-millisecond scans on short text.

• Checklist: secrets from env, redaction on, tests green, health OK.

Redaction and detection must ship together; detection without safe output is incomplete. A common mistake is scanning too late or logging the secrets being caught. The tradeoff is coverage versus false positives.

• OutputGuard — scanning and redaction.

• SignatureVerifier — HMAC signing and verification.

• SecurityService — orchestration and metrics.

• scan() — findings, counts, redacted text.

• sign() / verify() — integrity operations.

• run_demo() — deterministic metric population.

• Chatbot egress filter: a support assistant redacts customer emails and card numbers from replies before display.

• Webhook integrity: a deploy service signs payloads with HMAC so receivers reject forged requests.