A blog by Allen Downey.
As of September 2018, I am moving Probably Overthinking It to a new location . Blogger has been an excellent host; so far, this blog has received more than two million page views. But earlier this month, when I published a new article, Blogger prompted me to post it on Google+, and I did. A few hours later I discovered that my Google+ account had been suspended for violating terms of service, but…
On Sunday (September 16, 2018) Eliud Kipchoge ran the Berlin Marathon in 2:01:39, smashing the previous world record by more than a minute and taking a substantial step in the progression toward a two hour marathon. In a previous article , I noted that the marathon record pace since 1970 has been progressing linearly over time, and I proposed a model that explains why we might expect it to…
The double dice problem Suppose I have a box that contains one each of 4-sided, 6-sided, 8-sided, and 12-sided dice. I choose a die at random, and roll it twice without letting you see the die or the outcome. I report that I got the same outcome on both rolls. 1) What is the posterior probability that I rolled each of the dice? 2) If I roll the same die again, what is the probability that I get…
Bungee jumping turns out to be more complicated than I realized. I use bungee jumping as an example in Modeling and Simulation in Python , which I am revising this summer. The book is not done, but you can see the current draft here . During the first phase of the jump, before the cord is fully extended, I treat the jumper as if they are in free fall, including the effect of gravity and air…
I am preparing a talk for the Joint Statistical Meetings ( JSM 2018 ) in August. It's part of a session called " Bringing Intro Stats into a Multivariate and Data-Rich World "; my talk is called "Inference in Three Hours, and More Time for the Good Stuff". Here's what I said I would talk about: Teaching statistical inference using mathematical methods takes too much time, emphasizes the least…
Almost two years ago, I had the pleasure of speaking at the inaugural meeting of the Boston Bayesians , where I presented " Bayesian Bandits from Scratch " ( the notebook for that talk is here ). Since then, the group has flourished, thanks to the organizers, Jordi Diaz and Colin Carroll. Last night I made my triumphant return for the 21st meeting, where I presented a talk I called "Bayesian Zig…
For most of my books, I provide a Python module that defines the functions and objects I use in the book. That makes some people angry. The following Amazon review does a nice job of summarizing the objections, and it demonstrates the surprising passion this issue evokes: 1.0 out of 5 stars Ruined by idiotic and unnecessary and MASSIVE complexity in stupidly designed custom code March 29, 2018…
I was recently interviewed by Sally Phelps, the Director of Postgraduate Planning at Olin. We talked about computer science at Olin, which is something we are often asked to explain to prospective students and their parents, employers, and other external audiences. Afterward, I wrote the following approximation of our conversation, which I have edited to be much more coherent than what I actually…
This is the fourth article in a series about changes in support for gun control laws over the last 50 years. In the first article I looked at data from the General Social Survey and found that young adults are less likely than previous generations to support gun control. In the second article I looked at data from the CIRP Freshman Survey and found that even the youngest adults, who grew up with…
This is the third article in a series about changes in support for gun control over the last 50 years. In the first article I looked at data from the General Social Survey and found that young adults are less likely than previous generations to support gun control. In the second article I looked at data from the CIRP Freshman Survey and found that even the youngest adults, who grew up with…
In their coverage of the Parkland school shooting, The Economist writes : Though polling suggests that young people are only slightly more in favour of gun-control measures than their elders, those surveys focus on those aged 18 and above. There may be a pre- and post-Columbine divide within that group. Based on my analysis of data from the General Social Survey (GSS) and the CIRP Freshman Survey,…
In current discussions of gun policies, many advocates of gun control talk as if time is on their side; that is, they assume that young people are more likely than old people to support gun control. This letter to the editor of the Economist summarizes the argument: It is unlikely that a generation raised on lockdown drills, with access to phone footage of gun rampages and a waning interest in…
This is the second in a series of articles related to computational science and education. The first article is here. The Six Stages of Computational Science When I was in grad school, I collaborated with a research group working on computational fluid dynamics. They had accumulated a large, complex code base, and it was starting to show signs of strain. Parts of the system, written by students…
I have written several books that use Python to explain topics like Bayesian Statistics and Digital Signal Processing. Along with the books, I provide code that readers can download from GitHub. In order to work with this code, readers have to know some Python, but that's not enough. They also need a computer with Python and its supporting libraries, they have to know how to download code from…
In the New York Time on Tuesday, John McWhorter argues that Donald Trump's characteristic speech patterns are not, a s some have suggested, evidence of mental decline. Rather, the quality of Trump's public speech has declined because, according to McWhorter: 1) "The younger Mr. Trump [...] had a businessman’s normal inclination to present himself in as polished a manner as possible in public…
Last week I had the pleasure of visiting UC Davis, where I co-led (along with Jason Moore ) a workshop on using computation in the STEM curriculum . We had about 20 participants, including faculty, staff, and graduate students from engineering, math, natural sciences and social sciences. Classes at UC Davis start today, so we appreciate the time the participants took from a busy week! We hope to…
This is an extended version of my article in the Scientific American blog . The data I used and all of my code are available in this Jupyter notebook . Secularization in the Unites States For more than a century religion in the the United States has defied gravity. According to the Theory of Secularization, as societies become more modern, they become less religious. Aspects of secularization…
Last night I had the pleasure of presenting a talk for the PyData Boston Meetup . I presented a project I started earlier this summer, using data from the General Social Survey to measure and predict trends in religious affiliation and belief in the U.S. The slides, which include the results so far and an overview of the methodology, are here: And the code and data are all in this Jupyter notebook…
In my Data Science class this semester, students worked on a series of reports where they explore a freely-available dataset, use data to answer questions, and present their findings. After each batch of reports, I will publish the abstracts here; you can follow the links below to see what they found. How Do You Predict Who Will Vote? Sean Carter One topic that enters popular discussion every four…
This article contains supporting material for this blog post at Scientific American . The thesis of the post is that modern programming languages (like Python) are qualitatively different from the first generation (like FORTRAN and C), in ways that make them effective tools for teaching, learning, exploring, and thinking. I presented a longer version of this argument in a talk I presented at Olin…
In my Data Science class this semester, students are working on a series of reports where they explore a freely-available dataset, use data to answer questions, and present their findings. After each batch of reports, I will publish the abstracts here; you can follow the links below to see what they found. The Impact of Military Status on Income Bracket Joey Maalouf Using the National Health…
In my Data Science class this semester, students are working on a series of reports where they explore a freely-available dataset, use data to answer questions, and present their findings. After each batch of reports, I will publish the abstracts here; you can follow the links below to see what they found. How do Europeans feel about Jewish, Muslim, and Gypsy immigration? Apurva Raman and Celina…
On these previous post about my favorite Bayes theorem problems , I got the following comment from a reader named Riya: I have a question. Exactly 1/5th of the people in a town have Beaver Fever . There are two tests for Beaver Fever, TEST1 and TEST2. When a person goes to a doctor to test for Beaver Fever, with probability 2/3 the doctor conducts TEST1 on him and with probability 1/3 the doctor…
Getting ready to teach Data Science in the spring , I am going back through Think Stats and updating the Jupyter notebooks. Each chapter has a notebook that shows the examples from the book along with some small exercises, with more substantial exercises at the end. If you are reading the book, you can get the notebooks by cloning this repository on GitHub , and running the notebooks on your…
Getting ready to teach Data Science in the spring , I am going back through Think Stats and updating the Jupyter notebooks. When I am done, each chapter will have a notebook that shows the examples from the book along with some small exercises, with more substantial exercises at the end. If you are reading the book, you can get the notebooks by cloning this repository on GitHub , and running the…