I recently decided to brush up on deep learning fundamentals, as I’m now deploying open-source transformer models for a webapp I made. (Shameless plug: it does text-to-speech for class readings… I made it for my partner who’s in grad school. You can see it here ). With that in mind, rather than (for example) go through the Karpathy videos or other existing curriculum, I decided to try to get…
I’m preparing to release an open-source package – the details don’t matter very much, but it’s a UX to fine-tune OCR models. As I’m prepping for release, I decide I should have a landing page. I am definitely on the AI coding assistant train (handing over my $20/month to anthropic, for claude code), and as such I make a vague request for it to give me something that isn’t just the default…
Several years ago, a friend working on his PhD asked if I could help digitize the Statistical Abstracts of the United States in order to spare him the hassle of going through these documents page-by-page. These abstracts are a historical Census Bureau dataset that includes many interesting things, like how many chickens were in the United States in 1920. If you want to know, there were ~360…
Starting June 3 and ending September 27, DDOT piloted two bus lanes on H and I streets. In particular, the District painted the rightmost lanes of these streets red, and dedicated those lanes to buses between 7AM to 9:30AM and 4PM to 6:30PM. But, did those shiny, fresh coats of paint actually do anything? According to DDOT, “Preliminary data comparing June 2018 and June 2019 found improvements in…
This past Fall, I matriculated at NYU CUSP . I’ve worked on a few cool projects this past semester, and wanted to share a couple of them. This first one is analyzing the impact of the Second Avenue Subway’s opening on real estate prices in Yorkville. Before I begin, I want to give special shout-outs to the best teammates ever, Sarah Schoengold and Hao Xi, who were an integral part of putting this…
At The Data Incubator , we really like anaconda. In particular, we like that we can quickly install and update environments that include the usual PyData libraries (numpy, scipy, pandas) at any version from binary code. One issue we’ve had, however, is that freezing conda environments, currently, is machine-specific. In particular, there are subtle differences in versions of binary dependencies…
Jupyter is a fantastic tool that we use at The Data Incubator for instructional purposes. One perk of using Jupyter is that we can easily test our code samples across any language there’s a Jupyter kernel for. In this post, I’ll show you some of the code we use to test notebooks! First, a quick discussion of the current state of testing ipython notebooks: there isn’t much documented about the…
Anyone who’s ever tried to write a nontrivial application on Google App Engine has encountered at least seven* design decisions that have led to serious head-scratching moments. One of those happened to me about a month ago, while integrating Chef into our course at The Data Incubator . Our goal was to allow for one-click spinning up (on DigitalOcean ’s cloud) and monitoring of our Fellows’ course…
Jupyter is a fantastic tool that we use at The Data Incubator for instructional purposes. In particular, we like to keep our curriculum compartmentalized via Jupyter notebooks. It allows us to test our code samples across any language there’s a Jupyter kernel for* and keep things in one place, so our fellows don’t have to rifle through a wide variety of file formats before getting to the…
This one’s been a long time coming. I now have a blog. Woohoo! I will post about a variety of things. One of my favorite blog-functions is a repository for random bits of knowledge you wouldn’t keep track of otherwise. I have a good feeling that’s what you’ll mostly find here - stuff that’s not just germane, but also useful, to my interests - hopefully it’s at least interesting, if not useful, to…