Data Scientist, San Francisco
I've added a comment system! I know it's an odd thing to do, since this is a fairly inactive blog, but that's also what makes it a safe testing ground. I've wanted a comment system ever since I switched to a static blog, but it's hard to find one that meets all my desiderata. It has to be hosted, but inexpensive--I don't want to figure out how to serve it myself, and I don't want to pay a lot of…
This was originally posted to RWorks . Here’s another puzzle, from Henry Dudeney’s Perplexities column in Strand Magazine , January 1924. Arrange the ten digits, 1 2 3 4 5 6 7 8 9 0, in such order that they shall form a number that may be divided by every number from 2 to 18 without in any case a remainder. As an example, if I arrange them thus: 1 2 7 4 9 5 3 6 8 0, this number can be divided by…
Continuing my theme from last time of mental calculation and divisibility tricks that I didn’t learn in school, here’s a cool one: A number N is divisible by 11 if the alternating sum of its digits is divisible by 11. For example, take the number 2574. Its alternating sum is 2 − 5 + 7 − 4 = 0 . Since 0 is divisible by 11, so is 2574. Let’s check, in R. M = 2574 # check divisibility directly if ( M…
I’ve been working on another puzzle post, which will eventually show up here (but probably first on Rworks ). Because the puzzles I present come from older sources---in this case, Strand Magazine from 1924---I try to solve the puzzles using techniques that a puzzle-solver would have access to at the time—in other words, techniques other than brute force search via computer. While looking for such…
After having worked on The Twelve Coins Puzzle by hand, I got curious about implementing Dyson’s “elegant” oblivious solution, which I mentioned in that previous post. While a quick skim gave me the general gist of the approach—and it is elegant—a deeper read for details made me wonder how mathematicians ever got anything done at all, in the old days. The note is telegraphic, to say the least. But…
Photo by Marco Verch, CC-2.0. Source This is a notoriously hard problem that comes in many flavors. I don't know where it comes from originally, but it garnered quite a bit of attention from mathematicians in the mid-twentieth century. Apparently some versions of it may have even distracted scientists away from their defense research during WWII ! The problem was popular on both sides of the…
I was thinking about the Four Weights problem again recently. You might recall that puzzle poses the question: what are the four weights that can weigh any object x in the (integer) range 1 to 40 on a balance scale? A balance scale and four weights. Image Source The puzzle just asks you to find the values of the weights, which are ( 1 , 3 , 9 , 27 ) . But how do you weigh x —that is, with (an…
This puzzle is again from Dudeney's "The Worlds Best Puzzles", in the December 1908 Strand Magazine . It's another one from Bachet, who gave us The Four Weights Problem . A wine connoisseur has a wine-bin of eight compartments, containing 60 bottles of wine, as shown below: His dishonest employee steals 4 bottles and rearranges the remainder. The connoisseur notices that the bottles have been…
Here are a couple of puzzles from Henry Dudeney's article "The Best Puzzles with Coins," in the Strand Magazine , July 1909. These are maybe less interesting than the mathematical puzzles that John and I have been doing, but for whatever reason, they caught my eye. Kissing Coins # If I lay a penny flat on the table, how many other pennies can I place around it, every one also lying flat on the…
This is from Henry Dudeney’s “Perplexities” article in the March 1925 issue of The Strand Magazine : A man went into a bank with 1,000 silver dollars and 10 bags. He said, “Place this money, please, in the bags in such a way that if I call and ask for a certain number of dollars you can hand me over one or more bags, giving me the exact amount called for without opening any of the bags.” How was…
Here’s another puzzle from Henry Dudeney’s article “The World’s Best Puzzles”, The Strand Magazine , December 1908. According to Dudeney, this puzzle is originally from Problèmes plaisans et délectables qui se font par les nombres (Pleasant and delectable number problems), by French mathematician Claude Gaspar Bachet de Méziriac (1551-1636). [1] A balance scale and four weights. Image Source You…
I was browsing the December, 1908 issue of The Strand Magazine (it’s related to a hobby of mine), when I came across an article called “The World’s Best Puzzles”, by Henry Dudeney , who seems to have been the Martin Gardner of his day. Here’s a cool puzzle from that article, which according to Dudeney was first recorded by Alcuin , Abbot of Canterbury (735-804). I assume it’s from his manuscript…
Note: This post is best seen on my site, on a desktop/laptop. Mobile and RSS readers won't show you all the margin-y goodness. Because I can't leave a challenge alone, I added basic margin notes to this blog. The approach is basically from Tufte-CSS , and I now have both (unnumbered) margin notes ⊕ Following Tufte, I'll use margin note for unnumbered marginalia, and side note for numbered ones.…
For no particular reason, I started wondering if I could add sidenotes to my blog. I've always liked them better than endnotes, especially for longer documents. This led me to the more general question: Is there an eleventy template for sidenotes ? The answer is yes . But before plunging in, I decided to try a fresh blog first. My repository of medieval ghost stories (forked from an earlier repo)…
This is a followup to the Win Vector blog article Post-hoc Adjustment for Zero-Thresholded Linear Models , in which I showed how to use a one-variable GAM (e.g., a spline) to adjust a linear model in a problem space where outcomes are strictly nonnegative. If you haven’t read that article, I suggest you check it out, first. Some Background # The original motivation for the work we did here was to…
Simpson’s paradox is when a trend that is present in various groups of data seems to disappear or even reverse when those groups are combined. One sees examples of this often in things like medical trials, and the phenomenon is generally due to one or more unmodelled confounding variables. In the course of something else that I was working on, I was trying to come with an example of a logistic…
...or, what nowadays one might call "non-AI generative art." Over this Memorial Day weekend, I played with a few simple algorithms, inspired by an old Stack Exchange code golf . The intention here isn't to stomp on AI-based generative art, but just to remember the simple pleasures of creating pretty pictures with math. The details are all at the github repository , but I thought I'd share some…
I recently posted elsewhere about something silly ChatGPT did when we asked it about states' names. Admittedly, that particular question was merely on the level of cocktail party conversation, or a bar bet; but it does point out that you really should never trust anything it says, unless you already know the answer. Here's what happened when we asked it a statistics question: User What are some…
After migrating this blog to Eleventy, I made the comment on Micro.blog that Eleventy "is so much easier to wrap my arms around" than Jekyll was. That got me to wondering--why is that? After some thought, I came up with a few reasons: Newbie-friendly design and documentation that makes the process less opaque. Starter code that is simple to read, but that still illustrates useful functionality A…
My jekyll-generated website (served by Github Pages) failed to build a couple of weeks ago; I don't know why. I'm not enough of a front-end developer to diagnose the problem. After looking around for an updated template to move to, I eventually decided to give the Eleventy static site generator a try. So this is my new site, based on Eleventy's official starter blog template. It wasn't too hard to…
So, my husband typed this into ChatGPT 3.5: What are some English words that are 4 letters long, start with f, and end with k? According to ChatGPT, there was only one word that fit that criterion: fork . Upon prodding, it admitted that there were maybe a few more: Typing a similar query into DuckDuckGo ( 4 letter word starting with f ending with k ) produced a page full of links to various…
I have a new blog post up on the Win Vector blog: Detecting Data Differences Using the Sphering Transform . In this post, I discuss how to convert the problem of detecting changes in multivariate distributions to the simpler problem of detecting changes in univariate distributions. Please check it out.
Just created a handy little Rosetta stone of common data operations using both the pandas and polars libraries. Maybe you'll find it useful, too.
This article is a shortened version of a post from the Wallaroo Blog , originally written by Julio Barros and me. I'm posting the non-Wallaroo section of that article here, with permission, because I think it's a useful reference for A/B testing---one that I refer to myself. Hopefully, others find it helpful as well. How to A/B Test Machine Learning Models # A/B tests are a key tool of business…
I have a new article up on the Win-Vector blog, about the ξ ('XI') correlation coeffient that was recently introduced by Professor Sourav Chatterjee in his paper, “A New Coefficient of Correlation” . Unlike traditional correlation coefficients, ξ does not assume that the relationship between x and y is linear; in principle, it can be any functional relationship. While this is potentially useful, ξ…
I ran across this essay recently on the role of design standards for scientific data visualization. The author, Jure Triglav, draws his inspiration from the creation and continued use of the NYCTA Graphics Standards, which were instituted in the late 1960s to unify the signage for the New York City subway system. Image: A Case for Spaceships (Jure Triglav) As the author puts it, the Graphics…
So there's this article that's been making the rounds called "The 10 Least Stressful Jobs of 2013" ; perhaps you've read it. I don't normally bother with articles like that, but it came to my attention because some of my old graduate-school friends (who are professors) threw a mini-rant on social media over the fact that University Professor is the Number One least stressful job of the year,…
One of my favorite cheesy movies is a gem from 1984 called The Adventures of Buckaroo Banzai Across the 8th Dimension . For those who haven't seen it, Buckaroo Banzai is a brilliant young neurosurgeon and particle physicist who spends his days conducting cutting-edge research. At night, he and his research colleagues -- all engineers and scientists and doctors -- rock New Jersey as a band called…
I came across an interesting article in The Atlantic a little while back that discussed the connection between writing and thinking . New Dorp, a Staten Island high school in a poor and working-class neighborhood, was able to improve student performance when they realized that their students couldn’t write. These underperforming students often could read and could do math. The majority of them…
When people ask me what it means to be a data scientist, I used to answer, "it means you don't have to hold my hand." By which I meant that as a data scientist (a consulting data scientist), I can handle the data collection, the data cleaning and wrangling, the analysis, and the final presentation of results (both technical and for the business audience) with a minimal amount of assistance from my…
I came across a post from Emily Willingham the other day: "Is a PhD required for Good Science Writing?" . As a science writer with a science PhD, her answer is: is it not required, and it can often be an impediment. I saw a similar sentiment echoed once by Lee Gutkind, the founder and editor of the journal Creative Nonfiction . I don't remember exactly what he wrote, but it was something to the…