As part of my work on refstudio , I spent some time prototyping a chatbot that could answer questions about a corpus of PDFs. Tools like LangChain , LlamaIndex , Haystack , and others all have built-in abstractions to simplify this task, but I find that building a simplified version from scratch helps me …
FIT - Flexible and Interoperable Transfer - is a protocol designed for storing and sharing data from fitness and health devices. Since getting a Coros running watch in July 2022, I've been exporting the FIT file data to Dropbox after every run. Having all of this data laying around seemed like a …
In my all my years of working in Python, I don't think I've ever had to create a standalone executable. But, it finally happened. It was a pretty seamless experience, but I did hit a minor hiccup, so I wanted to capture some notes for my future self (and others …
I’ve wanted to get in the habit of writing more, so I’m taking some inspiration from old school blogs and sharing some things I've recently enjoyed. Enjoy your weekend! [Music] Spectrum by Max Cooper I'm very into synthesizer centric music. I'm also into music that creates a sonic …
I had a heart attack earlier this year. A "widow maker" heart attack . I fell into cardiac arrest and was clinically dead for a couple of minutes. I'm incredibly fortunate two nurses happened to be nearby. They defibrillated me and administered CPR until the ambulance arrived and took me to …
Despite regularly wanting to write more, I haven’t written anything in 18 months. I have countless drafts that I can’t seem to finish, or at least get into a place I feel comfortable publishing. I've mentally blocked myself from doing so. I can think of plenty of reasons …
The other day I spent far too much time trying to figure out how to mock a module-level function that was being used inside of my class's method in Python. My googling didn't lead to obvious answers, so I figured it'd be good to document here for future reference. Imagine …
Sometimes I'm expecting an email and want to be notified shortly after receiving it. But I also don't want to stare at my inbox, something I don't particularly enjoy checking in the first place. To illustrate an example, imagine you're browsing a niche site with limited edition goods, some of …
A while back I wrote about deploying my site using Github and Travis CI. But recently it seems Travis CI stopped being free for open source projects . If you're using a static site generator for your site and hosting it on it on S3, you can use Github Actions to …
In 2014, I wrote a custom theme for Pelican , the static site generator I use for this site. At the time, there were few themes available and I wanted something that was fairly simple in its design, but also that I understood well enough to tweak as necessary. I opted …
This is part of a series of posts I have written about web scraping with Python. Web Scraping 101 with Python , which covers the basics of using Python for web scraping. Web Scraping 201: Finding the API , which covers when sites load data client-side with Javascript. Asynchronous Scraping with Python …
I tend to forget how to write certain blocks of code when I haven't written them in a while. Here's a common machine learning preprocessing task that falls into that category. Imagine you have some event logs that capture an entity ID (user, store, ad, etc), timestamp, an event name …
In 2015, former Major Leaguer Lenny Dykstra went on Colin Cowherd’s radio show and claimed that he used to hire private investigators to find dirt on umpires. The intention of doing so was to turn that dirt into a more favorable strike zone for himself. You can find the …
Chicago's a big city that feels small -- everyone seems only a degree or two away from one another. This feels especially true within Chicago's tech and data science communities. As a result, I occasionally get asked about hiring data scientists. Specifically, how do you vet, hire, and evaluate a data …
Every so often, data scientists who are thinking about going off on their own will email me with questions about my year of freelancing (2015). In my most recent response, I was a little more detailed than usual, so I figured it'd make sense as a blog post too. If …
A while back (July 2015), I was fortunate to speak at PyData in Seattle about the pitfalls of overreliance on data. You can find the slides here . My talk centered around my belief that the term "data-driven," when taken at its face value, should not be something we strive for …
This is part of a series of posts I have written about web scraping with Python. Web Scraping 101 with Python , which covers the basics of using Python for web scraping. Web Scraping 201: Finding the API , which covers when sites load data client-side with Javascript. Asynchronous Scraping with Python …
This year's National League Cy Young race was pretty much a toss-up, with each of Jake Arrieta , Zack Greinke , and Clayton Kershaw putting up numbers we haven't seen in a decade or more. By now we know that Arrieta wins the award, but being the Cubs homer I am, I …
Despite having done it countless times, I regularly forget how to build a cohort analysis with Python and pandas . I’ve decided it’s a good idea to finally write it out - step by step - so I can refer back to this post later on. Hopefully others find it useful …
I’ve had a bunch of beer reviews and ratings data sitting on my hard drive for about year. For a beer nerd like me, that’s a pretty cool dataset, yet I’ve let it collect digital dust. Fast forward to last week, where somehow I wound up in …
Update: As of December 2020, Travis CI has stopped being free for open source projects. If you've been using Travis to deploy your static site, I recommend migrating to Github Actions. I've written a post about how to do so here . I’m an unabashed supporter of “Keep It Simple …
This is part of a series of posts I have written about web scraping with Python. Web Scraping 101 with Python , which covers the basics of using Python for web scraping. Web Scraping 201: Finding the API , which covers when sites load data client-side with Javascript. Asynchronous Scraping with Python …
A few weeks ago, I gave a pandas tutorial at PyData NYC titled "Translating SQL to pandas. And back." I don't remember why I put the "And back" in there - if you can translate things one way, you can translate them the other way, too. Anyway, here's the abstract: SQL …
Recently, I've been listening to a lot of lo-fi rock band, Cloud Nothings . Their album, Here & Nowhere Else , has been critically lauded , including garnering "Best New Music" from Pitchfork . As a result, when they came to Chicago's tiny Lincoln Hall in May, tickets sold out in a hurry - well before …
Data analysis is hard. What makes it hard is the intuitive aspect of it - knowing the direction you want to take based on the limited information you have at the moment. Additionally, it's communicating the results and showing why your analysis is right that makes this all the more difficult …
"We're talking …
I love college basketball. A lot. My beloved Marquette Golden Eagles are probably the only sports team that can really put me in agony. Last season, I was watching a nail-biter against Notre Dame . With 3:36 left, Notre Dame's Jack Cooley grabbed an offensive board and promptly passed out …
UPDATE: If you're interested in learning pandas from a SQL perspective and would prefer to watch a video, you can find video of my 2014 PyData NYC talk here . This is part three of a three part introduction to pandas , a Python library for data analysis. The tutorial is primarily …
UPDATE: If you're interested in learning pandas from a SQL perspective and would prefer to watch a video, you can find video of my 2014 PyData NYC talk here . This is part two of a three part introduction to pandas , a Python library for data analysis. The tutorial is primarily …
UPDATE: If you're interested in learning pandas from a SQL perspective and would prefer to watch a video, you can find video of my 2014 PyData NYC talk here . A while back I claimed I was going to write a couple of posts on translating pandas to SQL. I never …
I spent some time last weekend making minor changes to this site. Specifically: New typography - headers are Droid Serif , while everything else is Droid Sans . Fonts are also a bit bigger (I think it's easier to read). Added Jake Vanderplas' liquid tags plugin for Pelican, which allows for easy embedding …
Imagine you have a 4.2GB CSV file. It has over 12 million records and 50 columns. All you need from this file is the sum of all values in one particular column. How would you do it? Writing a script in python / ruby / perl /whatever would probably take a …
A few weeks back, I was talking with my friend Molly about personal domains and realized that her nickname, Bierface, was available. The exchange basically went like this: Me: I should buy bierface.com and just put up a ridiculous picture of you. Molly: You would have to do a …
Last weekend, I came across Jeff Atwood 's excellent visual explanation of SQL joins on Hacker News. It reminded me of teaching SQL to the incoming batch of PwC FTS associates a few years ago. Not many of them had prior programming experience, much less SQL exposure, so it was …
This is a follow-up to my previous post about web scraping with Python . Previously, I wrote a basic intro to scraping data off of websites. Since I wanted to keep the intro fairly simple, I didn't cover storing the data. In this post, I'll cover the basics of writing the …
The other week, I wrote a very basic intro to web scraping with Python . Some friends knew that I had experience scraping data and they wanted to learn, so I figured it would be a great opportunity to write something publicly and test how well I could explain it. I'll …
This is part of a series of posts I have written about web scraping with Python. Web Scraping 101 with Python , which covers the basics of using Python for web scraping. Web Scraping 201: Finding the API , which covers when sites load data client-side with Javascript. Asynchronous Scraping with Python …
I wrote a three part pandas tutorial for SQL users that you can find here . UPDATE: If you're interested in learning pandas from a SQL perspective and would prefer to watch a video, you can find video of my 2014 PyData NYC talk here . For some reason, I've always found …
So I finally got around to putting something of my own up. My intentions are mainly to use this space as a way to document mini projects that I'm working on, so plan on it being pretty programming, data, visualization, and statistics heavy - I get bored if I'm not learning …