This summer I’ve been competing in the SLICED machine learning competition, where contestants have two hours to open a new dataset, build a predictive model, and be scored as a Kaggle submission. Contestants are graded primarily on model performance, but also get points for visualization and storytelling, and from audience votes. Before SLICED I had almost no experience with competitive ML, so I…
Previously in this series The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The “birthday paradox” puzzle The “Spelling Bee honeycomb” puzzle Feller’s “coin-tossing” puzzle The “spam comments” puzzle The “prisoner coin flipping” puzzle I love 538’s Riddler column, and I’ve enjoyed solving the…
Previously in this series The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The “birthday paradox” puzzle The “Spelling Bee honeycomb” puzzle Feller’s “coin-tossing” puzzle The “spam comments” puzzle I love 538’s Riddler column, and I’ve enjoyed solving the May 1st puzzle . I’ll quote: You are…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The “birthday paradox” puzzle The “Spelling Bee honeycomb” puzzle Feller’s “coin-tossing” puzzle I love 538’s Riddler column, and the April 10 puzzle is another interesting one. I’ll quote: Over the course of three days,…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The “birthday paradox” puzzle I have an interest in probability puzzles and riddles, and especially in simulating them in R. I recently learned about Feller’s coin-tossing puzzle , from the book Mathematical Constants by…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The “birthday paradox” puzzle I love 538’s Riddler column, and the January 3 puzzle is a fun one. I’ll quote: The New York Times recently launched some new word puzzles, one of which is Spelling Bee. In this game, seven…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle The “largest stock profit or loss” puzzle The birthday problem is a classic probability puzzle, stated something like this. A room has n people, and each has an equal chance of being born on any of the 365 days of the year. (For simplicity, we’ll ignore leap…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle The “knight on an infinite chessboard” puzzle I recently came across an interview problem from A Cool SQL Problem: Avoiding For-Loops . Avoiding loops is a topic I always enjoy reading about, and the blog post didn’t disappoint. I’ll quote that post’s description of the problem: You have a table of trading…
Previously in this series : The “lost boarding pass” puzzle The “deadly board game” puzzle I’ve recently been enjoying The Riddler: Fantastic Puzzles from FiveThirtyEight , a wonderful book from 538’s Oliver Roeder. Many of the probability puzzles can be productively solved through Monte Carlo simulations in R. Here’s one that caught my attention: Suppose that a knight makes a “random walk” on an…
I recently came up with the idea for a series of screencasts: I've thought about recording a screencast of an example data analysis in #rstats . I'd do it on a dataset I'm unfamiliar with so that I can show and narrate my live thought process. Any suggestions for interesting datasets to use? — David Robinson (@drob) October 6, 2018 Hadley Wickham had the great suggestion of analyzing a Tidy…
Like a lot of people, I was intrigued by “I Am Part of the Resistance Inside the Trump Administration” , an anonymous New York Times op-ed written by a “senior official in the Trump administration”. And like many data scientists, I was curious about what role text mining could play. Ok NLP people, now’s your chance to shine. Just spitballing here but TF-IDF on “the op-ed” compared to the published…
A very useful concept in software engineering is technical debt . Technical debt occurs when engineers choose a quick but suboptimal solution to a problem, or don’t spend time to build sustainable infrastructure. Maybe they’re using an approach that doesn’t scale well as the team and codebase expand (such as hardcoding “magic numbers”), or using a tool for reasons of convenience rather than…
In January, I was excited to make an announcement about a shift in my career: I have some exciting news: today I'm joining @DataCamp as their Chief Data Scientist 🎉📊📈 pic.twitter.com/wiN9J4qSjx — David Robinson (@drob) January 29, 2018 When I first discussed the role with the DataCamp CEO, I described my goal as to “Make DataCamp as good at doing data science as we are at teaching it.”…
My new office introduced me to a betting game I wasn’t previously familiar with: Super Bowl squares . It’s played with a ten-by-ten grid, like this one from printyourbrackets.com : Each row and column gets an assortment of digits from 0-9 representing each team’s score, and each person playing the game (after putting in some money) adds their initials to one of the boxes. Then the winner is picked…
In a recent post , I offered a definition of the distinction between data science and machine learning: that data science is focused on extracting insights, while machine learning is interested in making predictions. I also noted that the two fields greatly overlap: I use both machine learning and data science in my work: I might fit a model on Stack Overflow traffic data to determine which users…
When I introduce myself as a data scientist, I often get questions like “What’s the difference between that and machine learning?” or “Does that mean you work on artificial intelligence?” I’ve responded enough times that my answer easily qualifies for my “rule of three”: When you’ve written the same code 3 times, write a function When you’ve given the same in-person advice 3 times, write a blog…
Last week I shared a thought on Twitter: When you’ve written the same code 3 times, write a function When you’ve given the same in-person advice 3 times, write a blog post — David Robinson (@drob) November 9, 2017 Ironically, this tweet hints at a piece of advice I’ve given at least 3 dozen times, but haven’t yet written a post about. I’ve given this advice to almost every aspiring data…
For the last few years I’ve been encouraging a particular approach to R education, teaching the dplyr and ggplot2 packages first and introducing real datasets early on. This week I’m excited to announce the next step: the release of Introduction to the Tidyverse , my new interactive course on the DataCamp platform. The course is an introduction to the dplyr and ggplot2 packages through an analysis…
Imagine you were going to a party in an unfamiliar area, and asked the host for directions to their house. It takes you thirty minutes to get there, on a path that takes you on a long winding road with slow traffic. As the party ends, the host tells you “You can take the highway on your way back, it’ll take you only ten minutes. I just wanted to show you how much easier the highway is.” Wouldn’t…
A year ago today, I wrote up a blog post Text analysis of Trump’s tweets confirms he writes only the (angrier) Android half . My analysis, shown below, concludes that the Android and iPhone tweets are clearly from different people, posting during different times of day and using hashtags, links, and retweets in distinct ways. What’s more, we can see that the Android tweets are angrier and more…
