Event-driven microservices — user, content, and notification services communicating through Kafka domain events with transactional outbox.
Schema governance plane — Avro catalog, compatibility checks, registry integration, lifecycle promotion.
Stream analytics — ksqlDB trending hashtag tables with Python fallback aggregation.
Production readiness console — blue-green deployment, multi-region failover, capacity planning from real host metrics.
Unified dashboard at http://localhost:5120 — live WebSocket console across all four planes.
Weeks 1–11 built ingestion, processing, Connect pipelines, Kafka Streams, and the operations security layer. This capstone assembles the production launch plane: how StreamSocial ships features without breaking mobile clients, surfaces viral trends in seconds, and survives region failure. Twitter’s Fleets rollout, Instagram’s schema-gated API changes, and LinkedIn’s multi-region Kafka mesh all share this same four-layer pattern — domain events, governed schemas, stream SQL, and controlled deployment.
Each write produces an immutable event (UserRegistered, PostCreated, PostLiked). Downstream services react asynchronously — the same choreography Uber uses between trip, payment, and notification domains. The outbox pattern guarantees the database commit and event intent share one transaction.
Content service caches users from user-events before accepting posts. A 200ms gap between registration and first post is normal; designing for it prevents phantom 404s — how Shopify tolerates catalog lag across regions.
BACKWARD compatibility means new consumers read old data. Adding display_name with a default passes; adding a required field without default fails. Confluent Schema Registry enforces this at the wire — the guardrail that kept Pinterest’s mobile feed stable across 400+ schema versions.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.