In a previous post , I explained that the pivot operation on a table of data is equivalent to a groupby operation on that data, but with the output table rearranged into a squatter configuration. But pivot tables are more than a groupby operation on tabular data! They're more, even, than a pivot operation on tabular data. That's because pivot tables — as they're actually used in…
One of my favorite public datasets is the baby name data published every year by the United States Social Security Administration (SSA). I like to do little analyses of this data in Row Zero from time to time, such as my earlier exploration of the popularity of baby names by final letter . My latest project is a report on which baby names had the most partisan political valence in 2024 — the…
A wonder of the modern world is that we all have tiny computers in our pockets that can quickly retrieve vast quantities of information. Because of this, it's common to assume that if something is knowable, it can be known simply by pulling out this computer and typing in the right question. A decade ago, the place one typed this question was a search engine called "Google". I was in graduate…
You find a lot of weirdness when you try to implement an Excel compatible spreadsheet. Filters, in particular, have got to be one of the weirdest spreadsheet features. Today, I learned that the order in which Excel presents items in its filter control violates transitivity, and hence is not even a proper partial order. How does Excel sort (numeric) filter options? Consider an Excel sheet with the…
Microsoft Excel stores numbers in a binary floating-point format. Specifically, the documentation tells us that Excel "was designed around" IEEE 754 and uses a version of the binary64 type specified in that document. In the course of building the world's fastest spreadsheet , I've had occasion to look into some of the nuances of how Excel handles numbers. In this post, I will explain one…
Three lanes for pickup trucks, lift to the sky, Seven for SUVs, eyes on the phone, None for the cyclists, doomed to die, One for our hero, speeding alone. He's the flow of traffic, don't impede that guy. One law to rule them all, one law to mind them, One law to bring them all and in the right lane bind them. He's the flow of traffic, don't impede that guy.
I first heard the term "pivot table" in, I think, 2012. At the time, I had no idea what a pivot table was. I made some desultory efforts to understand what they are and how one uses them, but I found Excel's interface inscrutable. Excel's documentation and Google were no help, either. Even today all you get is a bunch of "pivot tables are useful for aggregating data to make decisions"…
This is the story of my most fondly remembered project from my time on the S3 storage team in AWS. When I joined S3 in 2018, my team owned a service that was just a high-throughput, in-memory cache. When telling people about it they would ask, "Why not redis or memcached?" The answer was that this service (let's call it S3Cache) was as old as S3, i.e., about as old as memcached and older than…
Working in tech in Seattle, I have had a lot of colleagues from India, China, and several other countries. Almost all of them spoke English as a second language. This caused communication difficulties in at least one case I vividly remember (because it was my fault), and communication oddities in many more. One fun example was when one of my Indian colleagues told me that a meeting had been…
One of my favorite public datasets is the baby name data published every year by the United States Social Security Administration (SSA). When I joined Row Zero , one of the first things I did was gather this data into a single csv (it's published as one csv per year) and upload it to S3 for us to use as a test dataset. It's nice because it's not very big — less than 8 KiB gzipped —…