Lesson 53: Distance Filtering — Why H3 Lies and How Haversine Saves You

H3 at Resolution 9 gives you hexagons with an average edge length of ~174m and an area of ~0.105 km². When you execute a k-ring(cell, 1) query, you get 7 cells (the origin + 6 neighbors). k-ring(cell, 2) yields 19 cells.

Here is what every engineer misunderstands: H3 cells are not circles. They are irregular hexagons tiling a sphere. Two drivers at distance 190m can be in adjacent cells (ring 1) while a third driver at 140m can be in a non-adjacent cell depending on the orientation of the hexagon grid relative to the rider’s position. H3 ring index is a topological hop count, not a Euclidian distance bound.

This is not a bug. It is the explicit trade-off H3 makes: approximate spatial indexing in O(1) to enable prefix-scan-friendly key design in RocksDB. The lesson here: H3 prunes the candidate set. Haversine ranks it.

A standard Kafka Streams implementation using the DSL would look something like this:

This is a disaster. Here’s the physics of why:

Problem 1 — Full KTable join is O(N). The DSL’s KStream-KTable join iterates the entire driver table for every rider event. At 50,000 online drivers and 3,000 rider events/sec, you perform 150,000,000 Haversine calls per second on a single StreamThread. The process-latency-avg metric climbs from 2ms to 800ms within seconds. The consumer group falls behind. Kafka triggers a rebalance. You are now debugging a thundering herd at 2AM.

Problem 2 — Haversine is CPU-bound transcendental math. Each call requires Math.asin, Math.sqrt, Math.sin, Math.cos — all floating-point operations. Doing this unfiltered on the StreamThread (which also runs your poll() loop) starves Kafka’s heartbeat sender. The broker marks the consumer dead. Another rebalance.

Problem 3 — No spatial locality. The DSL join has no concept of “drivers near this rider.” It scans drivers in key-hash order — completely random spatial access. You cannot exploit RocksDB’s block cache locality because there is no spatial clustering in your key design.