The Integration Layer: Why MCPs “Blow Up” Context I wrote this post primarily to dig into why the Model Context Protocol (MCP) is often blamed for consuming massive context windows and driving out-of-control token costs in online discussions. The main complaint I see of MCP centers on its tendency to create massive context windows, leading to higher costs and increased latency. This isn’t…
Introduction With many real-time systems, there’s a pretty standard shape: ingest from a high-volume event stream assign those events to some entity periodically recompute derived state from recent observations At small to medium scale, a shared cache (Redis/Valkey) works great as the boundary between ingestion and aggregation. However, at a large enough scale, this same cache configuration can…
Context One of the more common failure modes in real-time geospatial systems has nothing to do with algorithms being wrong . It’s that the real world refuses to stay still. This post walks through a practical pattern for handling high-volume, real-time location data where part of your pipeline is explicitly geo-constrained (map matching, spatial joins, routing lookups, etc.), while the rest of the…
Handling Noisy Probes and Unrealistic Paths This is a follow up to my last post; discussing issues I saw when map matching and some notes on improving against those problems. One issue that became increasingly obvious after running the matcher against real probe data is how noisy probes interact with dense road networks. In urban environments especially (as shown in above example), a single probe…
Introduction Over the past few weeks I’ve been digging into a small, self-contained map matcher written in Go. The goal was simple: explore whether it’s possible to build a lightweight Hidden Markov Model (HMM) map matching service that doesn’t rely on external routing engines, yet still handles high concurrency and good throughput for short GPS traces. The use case I had in mind is one where a…
Introduction I went to State of the Map US this past weekend in Richmond, VA and learned more about Protomaps which makes self-serving your own static map tiles dead simple. This is the area of the web mapping ecosystem I am furthest away from, so I often rely on 3rd party vendors (as do most), such as Mapbox or Maptiler. Protomaps makes it possible to control this component and generate tiles…
Introduction This post demonstrates two tasks. First, it shows how slow string parsing is with Python and why more efficient serialization patterns are necessary for packing and unpacking an array of numerical values. Secondly, it demonstrates two methods - one that works within a “contained” Python-specific ecosystem and one that is more flexible and language and package agnostic. The impetus for…
Introduction The purpose of this post is to document the pattern for utilizing the UNNEST pattern for multiple arrays. UNNEST can be useful for “exploding” a row with an array to multiple rows (where the row count is equal to the number of values in each array), each paired with the other desired information from the original row adjacent to the array. However, it is not immediately clear that the…
Introduction AWS has been incrementally releasing some geospatial functionality on Athena over the past few years. Athena engine version 2 released a series of major quality of life improvements ( see blog post ) that allowed for the assembly of geospatial primitive spatial types. With version 3, new operators allow for more advanced spatial operations to be performed and expressed just through…
Introduction About a year and a half ago or so, AWS announced support for custom Lambda images. This unlocks a number of new advantages - namely the ability to create images that have complex install requiremnents prior to runtime. A perfect example of this is running Selenium on a Lambda to screenshot a website in-browser. One can quickly imagine the use cases (and related conveniences) of a…