I'm Jeremy Nelson. A data engineer based in Chicago.
The best data engineers I know aren't the ones chasing every new tool. They're the ones who can explain why a 5-year-old pipeline still works and when it's actually worth rebuilding. This isn't about resisting change. It's about understanding the real cost of complexity. The Trap of "Modern" Data Stacks Every few months, there's a new "must-have" tool in data engineering. The hype cycle is…
In the world of data engineering, efficient data integration and manipulation are critical for businesses to extract valuable insights. BigQuery, a fully managed data warehouse solution provided by Google Cloud, offers a powerful tool called the MERGE statement. In this technical blog post, we will dive into the MERGE statement and discuss its benefits for data engineering tasks. To illustrate its…
A short while ago I published a rather technical post on the development of a python-based attribution model that leverages a probabilistic graphical modeling concept known as a Markov chain. This post elaborates on the mechanics of the underlying markov chain attribution model I built in Python. I realize what might serve as better content is actually the motivation behind doing such a thing, as…
AI Content Optimizer Web Application Client needed a way to efficiently conduct analysis over a number of different on-site metrics for their own SEO clients. The tool would be comprised of two halves: a FAQ content creator and a ranking website content auditor. They wanted a tool that achieved the following: Content Optimization For a given keyword, find the most common nGrams present across the…
Have you run into this error before? Have you thought...How annoying is this? Well tfenv to the rescue. Install, manage and use multiple major and minor versions of Terraform Core and never have a headache again. https://github.com/tfutils/tfenv This seems like something that Terraform should allow one to manage within Core itself, no? Or am I dreaming? I've run into similar things with kubectl…
Jay Garfield's latest book is a fascinating look at the arguments against the concept of the "self" and why it is a good thing that there isn't one. In the first half of the book, Garfield spends his time discussing what the concept of "no self" would actually mean, then subsequently tearing through each argument against the idea of "no self". No Self For the uninitiated, the concept of a self is…
Most services we use these days are dominated by “Big Tech”. Email, blogging, etc.. There is a good reason for this. They happen to be very convenient to use, very user-friendly, and frequently free! I’ve grown an appreciation, however, for “Small Tech”. Maybe you’re unfamiliar with this phrase. I’m not entirely sure this is an appropriate label either. There’s something to be said about services…