“Of course, investigators could not have known how underpowered their research was, as their training had not prepared them to know anything about power, let alone how to use it in research planning. One might think that after 1969, when I published my power handbook that made power analysis as easy as falling off a log, the concepts and methods of power analysis would be taken to the hearts of…
As you prepare your courses on quantitative methods, consider including a section on statistical power . Here are five readings I like and often recommend. Quantitative Political Science Research is Greatly Underpowered (Arel-Bundock et al.) This paper shows that low power is a real problem—“doing what others have done” does not guarantee a well-powered study for the true effect. Instead, power…
Motivation This is for macOS using the default zsh shell (used in macOS Catalina and later). The command prompt on macOS doesn’t look nice by default. Here’s my preferred adjustment, where the prompt shows just the current folder followed by > . Like this: my-folder > Here’s what I want the Terminal to look like on macOS (when I’m working on website/ ). And Here’s what I want it to look like in…
It’s helpful to think of statistical power as determined by the ratio , where is the treatment effect and SE is the standard error of the estimate. 1 1 Bloom (1995) has a really beautiful paper on this idea. It’s one of my favorites. I also have a preprint on the topic. I like this ratio because it makes two distinct tasks clear: create a treatment with some oomph and estimate it precisely. While…
Power Rules This post is based on this paper called “Power Rules.” We can think of statistical power as determined by the ratio , where is the treatment effect and SE is the standard error of the estimate. To reason about statistical power, one needs to make assumptions or predictions about the treatment effect and the standard error. And as data-oriented researchers, we often want to use data to…
We can think of statistical power as determined by the ratio , where is the treatment effect and SE is the standard error of the estimate. 1 To reason about statistical power, one needs to make assumptions or predictions about the treatment effect and the standard error. 1 Bloom (1995) has a really beautiful paper on this idea. It’s one of my favorites. In this post, I discuss ways that pilot data…
In Rainey and McCaskey (2021) , Kelly McCaskey and I offer a accessible and practical (re)introduction to Firth’s penalized maximum likelihood estimator that (1) corrects the small sample bias and (2) reduces the excessive variance of the usual maximum likelihood estimator. Below, I bookmark other references that might be helpful. I’m sure there are embarrassing omissions. If you see an omission,…
Background on arguing for a negligible effect I remember sitting in a talk while I was a graduate student, and the speaker said something like: “I expect no effect here, and, just as I expected, the difference is not statistically significant.” Of course, that’s not a compelling argument for a null effect. A lack of statistical significance is an absence of evidence for an effect; it is not…
I try to write everyday. 1 1 See lots of caveats below! By “writing,” I mean “pushing the paper closest to publication just a little bit closer.” I want to think about the next step on the journey to the published paper and do it. According to this loose definition of writing, it might involve data collection, data analysis, creating slides, or even writing and polishing text. It might involve…
Firth’s Logit I like Firth’s logistic regression model (Firth 1993) . I talk about that in Rainey and McCaskey (2021) and this Twitter thread . Kosmidis and Firth (2021) offer an excellent, recent follow-up as well. I’ll refer you to the papers for a careful discussion of the benefits, but Firth’s penalty reduces the bias and variance of the logit coefficients. Goals for Benchmarking In this post,…
A Paper This post turned out to be somewhat popular, so I’ve written up a more formal, careful description of the idea in a full-length paper. You can find the preprint “Power Rules” here . Background I’ve wrapped up the argument that you should pursue statistical power in your experiments. In sum, you should do it for you (not a future Reviewer 2) and you shouldn’t see confidence intervals nestle…
Background When I give students formula for confidence intervals, I find that students don’t have a sharp concept of how those confidence intervals work—even if I explain the components of the formula well. Even though they understand—seemingly very well—that the point estimate is noisy, they struggle to conceptualize that a confidence interval can often include values on the incorrect side of…
A Paper This post turned out to be somewhat popular, so I’ve written up a more formal, careful description of the idea in a full-length paper. You can find the preprint “Power Rules” here . Background In this post, I address confidence intervals that are nestled right up against zero. 1 These intervals indicate that an estimate is “barely” significant. I want to be clear: “barely significant” is…
A Paper This post turned out to be somewhat popular, so I’ve written up a more formal, careful description of the idea in a full-length paper. You can find the preprint “Power Rules” here . Background There’s been some really good work lately on statistical power. I’ll point you to two really great papers. Arel-Bundock, Vincent, Ryan C. Briggs, Hristos Doucouliagos, Marco Mendoza Aviña, and T.D.…