The alarm went off at 6:20am. I rolled out of bed, opened my laptop, and pulled up Grafana. Equity markets open at 6:30 and I hoped last night’s fixes would hold. I watched request and order counts, comparing them to a week ago when things ran fine. Slack was already buzzing with engineers also checking metrics. The clock flipped to 6:30. The first metric tick came in, spiking the graphs. Okay,…
In my previous post I argued that the existing tools available to us as developers largely either optimize for developer velocity or scale, particularly when build backend applications. I postulate that if we had a framework that derived as much as possible from a schema definition plus associated metadata, that framework could optimize for both. To enable this, we first need to have a better way…
I’ve been working at Robinhood for a number of years since the early days and one phenomenon I’ve been fascinated by is the dramatic decrease in developer velocity as both the engineering organization and production systems have scaled. Here’s an example inspired by real-world events building out the backend API for showing dividends in-app in 2014: We decide in the morning we…
I’ve been quite impressed with DigitalOcean’s App Platform service. The team there struck a good balance of providing some extensibility while not going so far as to undermine the core value proposition of the service: simplicity. As a quick example: this blog is written using Hugo . It’s hosted on GitHub, and using the DigitalOcean admin console to start hosting the blog was a…
Interfacing with older APIs that don’t support unicode can be a pain. Too often I’ve seen errors from third parties saying: Error: characters must be in the range 0 to 127. Suppose that we’re writing an API for some site that internally will be interacting with these types of third parties. This API will allow people to create users and sign up. We want to allow people to sign up…