Week 9 : Kafka Streams Fundamentals & Stateless Processing (Days 41–45)

• A real-time engagement scoring topology with local state and tumbling windows (Day 41).

• A KStream feed-ranking pipeline that enriches each interaction into a ranking signal (Day 42).

• A content moderation stream using filter, map, and flatMap (Day 43).

• Sliding-window trending hashtag detection with velocity-based scoring (Day 44).

• A user reputation KTable with materialized views and tier labels (Day 45).

• One integrated dashboard on port 5090 where Run Demo drives all five pipelines from a single click.

Weeks 1–2 taught consumers and producers. Weeks 3–8 moved data in and integrated databases. Week 9 is where StreamSocial starts thinking in topologies—not single message handlers, but graphs of operators that scale with partitions.

Twitter’s timeline mixer, Netflix’s engagement counters, and Discord’s presence systems all compile down to the same primitives you build this week: stream → transform → aggregate → materialize.

Kafka Streams assigns one task per partition. Your topology’s operators run locally on that partition’s data—no cross-partition shuffle for stateless maps. Day 41’s engagement scorer teaches why partition key = content_id keeps ordering per post, the same reason Uber keys trips by ride_id.

Day 42 treats each like, share, and comment as an immutable event. You map it to a RankingSignal without remembering prior state. Feed rankers at Meta and Pinterest use this pattern for first-pass scoring before heavier ML joins.

Day 43 chains filter (spam), filter (policy), map (enrich), flatMap (mentions). The flatMap explosion cap (10 mentions) is a production guardrail—without it, one viral post spawns thousands of downstream records and stalls the task.

Day 44’s hopping windows let one hashtag sit in multiple overlapping buckets—capturing momentum spikes. Velocity (mentions now vs before) is why #WorldCup trends in minutes, not hours. Grace periods prevent cutting off late events.

Day 45 maintains latest reputation per user_id, backed by a changelog topic in real deployments. Compacted topics + KTable are how StreamSocial would serve “user tier” to moderation (Day 43) without hitting Postgres on every post.