Why are new startups growing so fast? Why is the wage distribution getting larger for software engineers? Why do I love infrastructure? Is open source dead? I think a lot of these things, and some more things, all have their roots in a big shift in how we build software. Or rather, how we buy software. Software development today is a lot about using the right vendors rather than building…
Writing code for a computer is hard enough. You take something big and fuzzy, some large vague business outcome you want to achive. Then you break it down recursively and think about all the cases until you have clear logical statements a computer can follow. Computers are very good at following logical statements. Now, let’s crank it up a notch. Let’s try to write code for humans ! I…
As I am en route to see my first total solar eclipse, I was curious how hard it would be to compute eclipses in Python. It turns out, ignoring some minor coordinate system head-banging, I was able to get something half-decent working in a couple of hours. I didn’t want to go deep on celestial mechanics, so I decided to leverage Python’s fantastic ecosystem for everything. The package…
CIA produced a fantastic book during the peak of World War 2 called Simple Sabotage . It laid out various ways for infiltrators to ruin productivity of a company. Some of the advice is timeless, for instance the section about “General interference with Organizations and Production”: Insist on doing everything through “channels”. Never permit short-cuts to be taken in order…
Long story short: I’m working on a super cool tool called Modal . Please check it out — it lets you run things in the cloud without having to think about infrastructure. Scaling out, scheduling, containerization, using GPUs, setting up webhooks, and all kinds of other stuff. It’s primarily meant for data teams. We aren’t quite live, but you can sign up for our waitlist. Context…
This is is in many respects a successor to a blog post I wrote last year about what I want from software infrastructure, but the ideas morphed in my head into something sort of wider. The genesis I encountered AWS in 2006 or 2007 and remember thinking that it’s crazy — why would anyone want to put their stuff in someone else’s data center? But only a couple of years later, I was…
Hi! It’s your friendly project management theorician. You might remember me from blog posts such as Why software projects take longer than you think , which is a blog post I wrote a long time ago positing that software projects completion time follow a log-normal distribution. Just a bit of a refresher if you don’t want to re-read that whole post. What does it mean that project…
Here’s a theory I have about cloud vendors (AWS, Azure, GCP): Cloud vendors 1 will increasingly focus on the lowest layers in the stack: basically leasing capacity in their data centers through an API. Other pure-software providers will build all the stuff on top of it. Databases, running code, you name it. We currently have cloud vendors that offer end-to-end solutions from the developer…
This isn’t as much of a blog post as an elaboration of a tweet I posted the other day: I think this specialization of data teams into 99 different roles (data scientist, data engineer, analytics engineer, ML engineer etc) is generally a bad thing driven by the fact that tools are bad and too hard to use — Erik Bernhardsson (@bernhardsson) July 21, 2021 This seem to have resonated with…
I guess I should really call this a parable. The backdrop is: you have been brought in to grow a tiny data team ( 4 people) at a mid-stage startup ( $10M annual revenue), although this story could take place at many different types of companies. It’s a made up story based on n-th hand experiences (for n ≤ 3), and quite opinionated. It’s a story about teams and organization, not the…
Software infrastructure (by which I include everything ending with *aaS, or anything remotely similar to it) is an exciting field, in particular because (despite what the neo-luddites may say) it keeps getting better every year! I love working with something that moves so quickly. In the last few months, I’ve thought a lot about where it’s going in the next 5-10 years and an wishlist…
I joined Better in early 2015 because I thought the team was crazy enough to actually change one of the largest industries in the US. For six years, I ran the tech team, hiring 300+ people, probably doing 2,000+ interviews, and according to GitHub I added 646,941 lines of code and removed 339,164. But I also got married, had two kids, bought an apartment and renovated it! From time to time, there…
It’s a popular attitude among developers to rant about our tools and how broken things are. Maybe I’m an optimistic person, because my viewpoint is the complete opposite! I had my first job as a software engineer in 1999, and in the last two decades I’ve seen software engineering changing in ways that have made us orders of magnitude more productive. Just some examples from…
I spent a ton of time looking at different software providers, both as a CTO, and as a nerd “advanced” consumer who builds stuff in my spare time. In the last 10 years, there has been an order of magnitude more products that cater directly to developers, through APIs, SDKs, and tooling. I’m pretty psyched about this trend. As the cost of building software goes down, that drives…
We live in a year of about 350,000 amateur epidemiologists and I have no desire to join that “club”. But I read something about COVID-19 deaths that I thought was interesting and wanted to see if I could replicated it through data. Basically the claim is that Sweden had an exceptionally “good” year in 2019 in terms of influenza deaths causing there to be more deaths…
Compensation has always been one of the most confusing parts of management to me. Getting it right is obviously extremely important. Compensation is what drives our entire economy, and you could look at the market for labor as one gigantic resource-allocating machine in the same way as people look at the stock market as a gigantic resource-allocating machine for investments. Yet, almost everything…
Hanlon’s razor is a classic aphorism I’m sure you have heard before: Never attribute to malice that which can be adequately explained by stupidity. I’ve found that neither malice nor stupidity is the most common reason when you don’t understand why something is in a certain way. Instead, the root cause is probably just that they didn’t have time yet . This happens all…
Let’s consider a toy model where you’re hiring for two things and that those are equally valuable. It’s not very important what those are, so let’s just call them “thing A” and “thing B” for now. For one set of abilities, the scatter plot looks like this: The assumption here is that A and B are drawn from a 2D-Gaussian with a mild positive…
I recently finished the excellent book Kochland . This isn’t my first interest in Koch—I read The Science of Success by Charles Koch himself a couple of years ago. Charles Koch inherited a tiny company in 1967 and turned it into one of the world’s largest ones. That’s impressive! Just a quick disclaimer just to get it out of the way. You may know the Koch brothers as the climate…
Just a quick note that my team is always hiring at Better . A lot of new people have been joining the team here in NYC lately—the tech team has actually grown from 35 to 60 in just ~3 months. We are primarily looking for senior software engineers and/or engineering managers. But we would love to talk if you have less experience too! Our main tech stack is mostly TypeScript and Python. We are also…
My company has a buffet every Friday, and the lines grow to epic proportions when the food arrives. I’ve suspected for years that the “classic” buffet line system is a deeply flawed and inefficient method, and every time I’m stuck in the line has made me more convinced. This is a hill I’m willing to die on, and it’s also a place where I have a secret weapon: a…
No one asked for this, but I’m something like ~12 years into my career and have had my fair share of mistakes and luck so I thought I’d share some. Honestly, I feel like I’ve mostly benefitted from luck. Some of the things I did on a whim turned out to be excellent choices many years later. Some of the things were clear blind spots in hindsight. If I could give my 12 years…
This is a blog post originally featured on the Better engineering blog . If you want to link to this article or share it, please go to the original post URL ! Separately, I’m sorry it’s been so long with no posts on this blog. Between kids, moving, and being a startup CTO, I’ve been busy. I have a few posts coming down the pipe though, so stay tuned… Lots of companies need…
Anyone who built software for a while knows that estimating how long something is going to take is hard . It’s hard to come up with an unbiased estimate of how long something will take, when fundamentally the work in itself is about solving something. One pet theory I’ve had for a really long time, is that some of this is really just a statistical artifact. I suspect devs are actually…
When I started building up a tech team for Better , I made a very conscious decision to pay at the high end to get people. I thought this made more sense: they cost a bit more money to hire, but output usually more than compensates for it. Many fellow CTOs, some went for the other side of the spectrum. This was a mystery to me, until it all made sense to me. What is output? Before we get started,…
A modern tech stack typically involves at least a frontend and backend but relatively quickly also grows to include a data platform. This typically grows out of the need for ad-hoc analysis and reporting but possibly evolves into a whole oil refinery of cronjobs, dashboards, bulk data copying, and much more. What generally pushes things into the data platform is (generally) that a number of things…
It started with a tweet: New years resolution: every plot I make during 2018 will contain uncertainty estimates — Erik Bernhardsson (@bernhardsson) January 7, 2018 Why? Because I’ve been sitting in 100,000,000 meetings where people endlessly debate whether the monthly number of widgets is going up or down, or whether widget method X is more productive than widget method Y. For almost…
This is a bit of a rant but I really don’t like software that invents its own query language. There’s a trillion different ORMs out there. Another trillion databases with their own query language. Another trillion SaaS products where the only way to query is to learn some random query DSL they made up. I just want my SQL back. It’s a language everyone understands, it’s been…
I get bored reading management books very easily and lately I’ve been reading about a wide range of almost arbitrary topics. One of the lenses I tend to read through is to see different management styles in different environments. It turns out that some truly f—ng horrific people have some smart management ideas. This is not maybe surprising. If you have some twisted goals, you can’t…
As some of you may know, one of my side interests is approximate nearest neighbor algorithms. I’m the author of Annoy , a library with 3,500+ stars on Github as of today. It offers fast approximate search for nearest neighbors with the additional benefit that you can load data super fast from disk using mmap. I built it at Spotify to use for music recommendations where it’s still used…
Ok, so I have to first preface this whole blog post by a few things: I really struggle with the term microservices . I can’t put my finger on exactly why. Maybe because the term is hopelessly ill-defined, maybe because it’s gotten picked up by the hype train. Whatever. But I have to stick to some type of terminology so let’s just roll with it. This blog post might be mildly…
I have done roughly 2,000 interviews in my life. When I started recruiting, I had so much confidence in my ability to assess people. Let me just throw a couple of algorithm questions at a candidate and then I’ll tell you if they are good or not! Over time I’ve come to the (slightly disappointing) realization that knowing who’s going to be good at their job is an extremely hard…
I’ve been reading up on operations research lately, including queueing theory . It started out as a way to understand the very complex mortgage process (I work at a mortgage startup ) but it’s turned into my little hammer and now I see nails everywhere. One particular relationship that turns out to be somewhat more complex is the relationship between cycle time and throughput . Here…
I started writing this blog in late 2012, partly because I felt like it would help me improve my English and my writing skills, partly because I kept having a lot of random ideas in my head and I wanted to write them down somewhere. I honestly never cared too much about finding a particular niche, I just wanted to write down stuff that I found interesting. I set up a Wordpress blog on my crappy…
UPDATE(2018-06-17): There are is a later blog post with newer benchmarks ! One of my super nerdy interests include approximate algorithms for nearest neighbors in high-dimensional spaces. The problem is simple. You have say 1M points in some high-dimensional space. Now given a query point, can you find the nearest points out of the 1M set? Doing this fast turns out to be tricky. I’m the…
I’m interrupting the regular programming for a quick announcement: we’re looking for data engineers at Better . You would be the first one to join and would work a lot directly with me. Some fun things you could work on (these are all projects I’m working on right now): Building a forecasting model using MCMC to predict volume the next few months. Productionize it in the form of…
Turns out having a toddler isn’t super compatible with reading. I used to read ~100 books/year as a teenager, but it has slowly deteriorated to maybe 20-30 books, at most. And I don’t even finish all of them because life is too short! Some books are just not that interesting. So what were some of the books worth mentioning? Economics / Venture capital / Misc I’m a sucker for…
I spent a few days during the holidays fixing up a bunch of semi-dormant open source projects and I have a couple of blog posts in the pipeline about various updates. First up, I made a number of fixes to Git of Theseus which is a tool (written in Python) that generates statistics about Git repositories. I’ve written about it previously on this blog. The name is a horrible pun (I’m a…
I spent six years at a company that went from 50 people to 1500 and one contributing factor leading to my departure was that I went from a “maker” to a person stuck in meetings every day. It wasn’t that I wanted to do that, but everyone else kept dragging me into meetings. There’s about 47 million blog posts about why meetings suck and I’m not going to pile more onto…
I had an interesting idea a few weeks ago, best explained through an example. Let’s say you’re running an e-commerce site (I kind of do ) and you want to optimize the number of purchases. Let’s also say we try to learn as much as we can from users, both using A/B tests but also using just basic slicing and dicing of the data. We are looking at how many people convert (buy our…
I’ve been a bit bad at posting things with a regular cadence lately, partly because I’m trying to adjust to having a toddler, partly because the hunt for clicks has caused such a high bar for me that I feel like I have to post something Pulitzer-worthy. But things are always cooking, so let’s break this pattern with a quick notice on something I’ve been working on! Annoy…
There are often close relationships between top level business metrics. For instance, it’s well known that retention has a super strong impact on the valuation of a subscription business. Or that the % of occupied seats is super important for an airline. A fun little toy model that I can up with generates a curious relationship between conversion rates and revenue. The intuition Let’s…
A funny thing about being a foreigner is how you realize people take broken things for granted. I’m going to go out on a limb here claiming that the US has a pretty dumb banking system . I could talk about it all day, but right now I want to focus on a very particular piece of it: how to verify your identity online. Of course, since the Equifax breach, people are freaking out about the fact…
Just for fun, I generated these graphs of the number of letters in the word for each number. I really spent about 10 minutes on this (ok…possibly also another 40 minutes tweaking the plots): More languages!! I love how Spanish has a few super compact words: “cien mil” for 100,000 for instance. Only eight letters, versus English “one hundred thousand” (20 letters). I…
Here’s a dumb extremely accurate rule I’m postulating* for software engineering projects: you need at least 3 examples before you solve the right problem . This is what I’ve noticed: Don’t factor out shared code between two classes. Wait until you have at least three. The two first attempts to solve a problem will fail because you misunderstood the problem. The third time…
I just bought Machine, Platform, Crowd: Harnessing Our Digital Future and discovered that it mentions my blog – in particular the post When machine learning matters . Ok, I lied a little bit. I didn’t discover it serendipitously. Someone actually emailed me saying I was mentioned, and so I ordered the book for same-day delivery. But I was seriously planning to read the book anyway – having…
There’s about 765 million blog posts about the diversity “memo” that leaked out of Google a couple of weeks ago. I think the case for any biological difference is pretty weak, and it bothers me when people refer to an “interest gap” as anything else than caused by the environment. Maybe because I have a daughter, maybe because I have too many female friends who told…
I just spent a few days in Italy, on the Ligurian coast. Even though we were on the west side of Italy, the Mediterranean sea was to the east, because the house was situated on a long bay. But zooming in even more, there were parts of the coast that were even more twisted – to the point where it had turned a full 360 degress so you ended up having the sea to the west again. Anyway, that made me…
I’ve written before about the importance of iterating quickly but I didn’t necessarily talk about some concrete things you can do. When I’ve built up the tech team at Better , I’ve intentionally optimized for fast iteration speed above almost everything else. What are some ways we did that? Continuous deployment My dubious claim is that we might be the only financial…
Remember when everyone had a really ugly blog with a blogroll ? Anyway, just think the word is funny. I follow a few hundred blogs using Feedly and Reeder and have been reading a few hundred thousand blog posts over the last 10 years. Here’s some stuff I think everyone should follow. Not going to share a million blogs, just a few top ones. That way you don’t have to think about it,…