From Complexity to Clarity
Data platform complexity is rarely something that happens to an organization. It is something the organization builds, one reasonable decision at a time. Many data platforms are not an architecture; they are the sediment of decisions nobody ever made. The first three parts dealt with problems arriving from outside, namely inherited organizational structures , misdirected incentives , and data that…
Every dataset is a map, which means it shows a territory in simplified form and leaves things out, and both of those are the point rather than the flaw. The first two parts dealt with structures and metrics , meaning with questions that arise before the analysis. This one starts later. Assume the organization works, the metrics are cleanly defined and the numbers are computed correctly. Wrong…
Goodhart s Law in data and AI describes a moment every data-driven organization walks into sooner or later: the moment a metric stops describing reality and starts producing it. That is the weakness this part covers, and it is a peculiar one, because it grows out of doing the right thing. Part 1 looked at organization and collaboration, where the problems at least announce themselves; this one…
The data warehouse of the 1990s, the data lake of the 2010s and today s GenAI platform fail surprisingly often for the same reasons. Three decades, three generations of technology, one set of mistakes. That is difficult to explain if the problem is the technology. My thesis for this series is therefore that data and AI initiatives rarely fail because of technology. They fail because of…
Big context, little content that is the short summary of what research tells us about large LLM context windows. Vendors advertise one million tokens as if capacity were the same as capability. Load everything in, the message goes, and the model will sort it out. Data professionals have heard this promise before. Fifteen years ago the container was called a data lake, and the currency was…
GenAI critical thinking is becoming one of the scarcest skills in knowledge work. GenAI makes it easier than ever to get answers. What it does not do is make it easier to question them. That asymmetry sits at the heart of a shift that affects not just data management, but every field where decisions depend on the quality of information. For years, I have been teaching data management topics at…
Open table formats like Apache Iceberg and Delta Lake are changing how companies store analytical data and who controls it. If you store your analytical data in a proprietary format, you re locked in to the engine, the tooling, the license costs. The data belongs to the company, not the engine. Analytical systems are increasingly moving to the cloud; data warehouses, lakehouses and BI platforms…
Data professional fundamentals have never been more valuable, yet for years the industry measured progress by volume: lines written, data products shipped, pipelines grown, dashboards created. AI has now exposed how little these metrics ever said about actual impact. The real cost of software was never in the writing. It was always in the thinking behind it . Delays were rarely caused by coding…
I m a passionate runner. And like many data enthusiasts, I can t resist collecting data about the things I care about. This post is both a personal reflection on my training and a hands-on tour of how Oracle can model very different questions from streets you run most often, to the feel of a run, to the network of your routes. There are six very different ways to think about your running data and…
In the rapidly evolving landscape of 2026, Data Tech Unlearning has become a critical survival skill for architects and leaders. In the data world, we often define ourselves by the complexity of the tools we’ve mastered. Someone who spent years taming Spark clusters might feel almost insulted by a system that runs perfectly fine on a single node with DuckDB. But that’s exactly the trap: complexity…